-- Logs begin at Wed 2022-08-24 11:53:38 UTC, end at Wed 2022-08-24 12:51:16 UTC. -- Aug 24 12:18:22.750949 dv-cinder-848900-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 12:18:22.759502 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 12:18:22.761550 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 12:18:22 2022] *** Aug 24 12:18:22.761550 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 12:18:22.761550 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 12:18:22.761550 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: nodename: dv-cinder-848900-5-iscsi Aug 24 12:18:22.761550 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: machine: x86_64 Aug 24 12:18:22.761550 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: clock source: unix Aug 24 12:18:22.761550 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: pcre jit disabled Aug 24 12:18:22.761550 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: detected number of CPU cores: 4 Aug 24 12:18:22.761550 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: current working directory: / Aug 24 12:18:22.761550 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: detected binary path: /usr/bin/uwsgi-core Aug 24 12:18:22.762296 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: your processes number limit is 47680 Aug 24 12:18:22.762296 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: your memory page size is 4096 bytes Aug 24 12:18:22.762296 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: detected max file descriptor number: 2048 Aug 24 12:18:22.762296 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: lock engine: pthread robust mutexes Aug 24 12:18:22.762296 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: thunder lock: enabled Aug 24 12:18:22.762296 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 12:18:22.762296 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 12:18:22.801071 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: Python main interpreter initialized at 0x561e0f6b39c0 Aug 24 12:18:22.801071 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: python threads support enabled Aug 24 12:18:22.801071 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: your server socket listen backlog is limited to 100 connections Aug 24 12:18:22.801071 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: your mercy for graceful operations on workers is 90 seconds Aug 24 12:18:22.801425 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 12:18:22.801507 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: *** Operational MODE: preforking *** Aug 24 12:18:22.801764 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: *** uWSGI is running in multiple interpreter mode *** Aug 24 12:18:22.801764 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: spawned uWSGI master process (pid: 81626) Aug 24 12:18:22.801958 dv-cinder-848900-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 12:18:22.803656 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: spawned uWSGI worker 1 (pid: 81637, cores: 1) Aug 24 12:18:22.803656 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: spawned uWSGI worker 2 (pid: 81638, cores: 1) Aug 24 12:18:22.804755 dv-cinder-848900-5-iscsi devstack@keystone.service[81626]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 12:18:27.149341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 12:18:27.149341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: from cryptography.utils import int_from_bytes Aug 24 12:18:27.150097 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 12:18:27.150097 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: from cryptography.utils import int_from_bytes Aug 24 12:18:27.150338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 12:18:27.150338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: from cryptography.utils import int_from_bytes Aug 24 12:18:27.150801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 12:18:27.150801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: from cryptography.utils import int_from_bytes Aug 24 12:18:27.200621 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 12:18:27.200809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 12:18:27.200986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] command line args: [] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 12:18:27.201258 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 12:18:27.201346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 12:18:27.201428 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ================================================================================ {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 12:18:27.201514 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 12:18:27.201633 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] command line args: [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 12:18:27.201713 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] admin_token = **** {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.201907 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] config_dir = [] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.201994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 12:18:27.202080 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ================================================================================ {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 12:18:27.202153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.202261 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] config_source = [] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.202347 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] admin_token = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.202445 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] debug = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.202555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] config_dir = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.202647 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.202953 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.202953 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] config_source = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.203137 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.203235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] debug = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.203342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.203445 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.203571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.203710 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.203813 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.203920 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.204030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] list_limit = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.204133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.204237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] log_config_append = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.204341 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.204450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.204450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] log_dir = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.204642 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.204737 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] log_file = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.204845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_config_append = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.204845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.205029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.205160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_dir = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.205237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.205361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_file = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.205450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.205531 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.205604 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.205736 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.205812 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.205930 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.206005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.206121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.206210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.206290 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.206365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.206489 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.206566 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.206694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.206779 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.206853 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.206969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.207051 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.207130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.207252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.207339 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.207413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.207529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.207645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.207822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.207904 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.208131 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.208345 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.208519 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.208645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] publish_errors = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.208783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.208906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.58/identity {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.209028 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.209149 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] publish_errors = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.209279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.209407 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.209535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.209663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.209784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.209923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.210055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] standard_threads = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.210188 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.210321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.210455 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.210587 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.210718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] standard_threads = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.210843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.210843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] use_journal = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.211082 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.211204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] use_json = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.211376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.211503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] use_stderr = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.211651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.211732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] use_syslog = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.211805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] use_journal = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.211887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.211970 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] use_json = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.211970 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] use_stderr = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.212102 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.212174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] use_syslog = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.212245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.212316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 12:18:27.212400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.212480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.212559 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.212632 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.212710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.212788 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.212872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.212945 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.213016 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.213088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.213159 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.213260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.213350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.213509 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.213596 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.213749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.213835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.213981 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.214071 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.214231 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.214317 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.214468 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.214548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.214702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.214793 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.214975 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.215122 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.215297 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.215383 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.215464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.215547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.215701 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.215784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.215941 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.216021 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.216178 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.216254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.216416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.216497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.216674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.216753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.216912 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.216994 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.217157 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.217236 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.217388 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.217468 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.217618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.217698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.217847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.217933 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.218083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.218163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.218327 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.218413 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.218559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.218646 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.218787 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.218865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.219016 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.219096 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.219259 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.219338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.219487 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.219566 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.219743 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.219826 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.219988 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.220068 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.220220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.220300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.220455 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.220542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.220687 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.220767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.220915 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.221000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.221143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.221224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.221370 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.221452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.221598 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.221683 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.221833 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.221907 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.222052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.222139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.222280 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.222358 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.222513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.222599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.222740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.222820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.222975 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.223055 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.223210 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.223289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.223442 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.223521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.223690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.223773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.224074 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.224203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.224310 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.224407 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.224502 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.224597 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.224701 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.224798 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.224901 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.225036 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.225125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.225208 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.225288 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.225369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.225488 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.225565 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.225725 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.225814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.225954 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.226035 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.226195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.226325 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.226434 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.226523 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.226664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.226756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.226895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.227016 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.227154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.227245 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.227370 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.227458 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.227609 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.227746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.227874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.227953 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.228117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.228216 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.228336 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.228414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.228574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.228664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.228808 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.228897 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.229055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.229176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.229290 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.229370 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.229521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.229614 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.229760 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.229840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.229994 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.230075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.230226 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.230315 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.230961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.231077 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.231201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.231313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.231433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.231684 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.231772 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.231911 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.231992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.232153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.232233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.232384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.232465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.232621 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.232702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.232857 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.232938 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.233093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.233175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.233325 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.233413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.233553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.233635 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.233822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.233966 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.234045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.234119 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.234224 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.234314 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.234452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.234542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.234682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.234761 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.234914 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.235038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.235153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.235242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.235386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.235466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.235639 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.235722 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.235887 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.235980 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.236126 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.236206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.236356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] auth.external = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.236437 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.236586 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.236683 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.236859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.236939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.237094 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.237173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.237323 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] auth.password = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.237436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.external = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.237550 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] auth.token = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.237644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.237785 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.237868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.238019 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.238100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.238263 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.238353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.password = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.238493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.238574 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] auth.token = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.238734 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.238813 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.238973 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.239063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.239199 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.239279 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.239435 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] credential.caching = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.239517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.239701 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.239797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.239948 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.240063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.240181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.240262 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.240416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.240504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.240658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.240740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.240894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.240976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.241142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.241230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.241380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.241462 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.241616 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.241695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.241858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.241945 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.242111 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.242193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.242363 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.242443 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.242600 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.242690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.242852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.242933 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.243105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.243187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.243342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] federation.caching = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.243432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.243577 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.243673 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.243845 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.243926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.244096 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.244190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.244327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.244408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.244563 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.244643 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.244797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.244878 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.245044 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.245136 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.245277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.245358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.245523 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.245606 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.245759 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.245844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.246006 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.246087 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.246819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.247016 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity.caching = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.247099 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.247306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.247420 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.247557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.247702 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.247827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.247946 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.248083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.248171 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.248334 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.248576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.248661 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.248737 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.248812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.248886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.249041 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.249132 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.249275 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.249358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.249517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.249599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.249768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.249858 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.249998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.250078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.250242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.250323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.250475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.250563 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.250708 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.250789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.250955 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.251037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.251197 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.251286 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.251446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.251526 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.251715 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.251798 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.251953 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.252047 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.252188 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.252270 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.252425 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.252506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.252654 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.252745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.252891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.252973 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.253130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.253210 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.253365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.253454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.253610 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.253691 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.253838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.253917 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.254082 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.254163 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.254329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.254408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.254561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.254640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.254805 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.254885 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.255050 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.255137 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.255279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.255515 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.255603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.255765 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.255848 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.256001 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.256083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.256235 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.256329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.256526 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.256606 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.256761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.256842 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.257000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.257089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.257239 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.257320 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.257481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.257563 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.257722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.257811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.257959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.258039 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.258205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.258287 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.258451 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.258541 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.258683 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.258764 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.258915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.258996 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.259149 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.259237 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.259381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.259463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.259640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.259724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.259880 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.259971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.260119 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.260202 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.260351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.260432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.260594 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.260685 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.260839 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.260964 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.261075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.261204 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.261307 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.261455 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.261537 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.261729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.261810 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.261985 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.262096 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.262224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.262329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.262458 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.262576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.262700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.262821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.262940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.263065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.263179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.263308 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.263414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.263552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.263692 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.263804 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.263944 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.264049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.264191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.264272 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.264439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.264522 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.264686 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.264767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.264922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.265003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.265170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.265253 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.265409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.265489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.265641 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.265721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.265878 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.265959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.266111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.266192 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.266356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.266438 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.266592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.266674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.266827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.266907 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.267059 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.267139 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.267302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.267382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.267539 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.267687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.267793 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.267875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.268028 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.268111 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.268264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.268343 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] resource.caching = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.268508 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.268588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.268750 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.268831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.268981 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.269061 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.269220 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.269299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.269463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.269546 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.269696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.269774 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.269941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.270030 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.270176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.270258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.270412 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.270493 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.270646 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] role.caching = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.270736 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.270889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.270969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] role.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.271116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.271196 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.271363 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.271519 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.271598 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.271739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.271856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] role.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.271937 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.272098 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.272190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] role.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.272327 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.272415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.272574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.272654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.272819 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.272908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.273064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.273145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.273292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.273372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.273536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.273626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.273780 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.273863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.274024 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.274106 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.274272 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.274361 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.274508 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.274665 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.274755 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.274902 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.274984 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.275151 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.275235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.275398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.275480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.275667 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.275869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.275964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.276112 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.276193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.276364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.276445 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.276615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.276696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.276853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.276934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.277084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.277165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.277317 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.277410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.277569 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.277695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.277820 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.277956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.278162 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.278244 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.278412 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.278494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.278645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.278725 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.278879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.278967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.279115 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.279231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.279357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] token.caching = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.279448 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.279594 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.279746 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.279861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.279942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.280098 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.280177 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.280333 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.280415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.280573 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.280654 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.280805 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.280884 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.281048 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.281130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.281286 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.281365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.281528 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.281613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.281762 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.281844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.281997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.282078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.282230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.282311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.282473 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.282556 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.282717 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.282798 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.282946 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.283027 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.283184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.283264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.283422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.283503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.283693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.283792 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.connection = **** {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.283948 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.284050 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.284181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.284274 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.284426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.284508 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.284669 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.284758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.284905 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.284987 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.connection = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.285141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.285231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.285374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.285455 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.285609 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.285698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.285853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.285934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.286088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.286169 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.286329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.286418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.286565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.286646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.286801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.286881 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.287033 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.287122 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.287275 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.287355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.287501 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.287580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.287764 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.287859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.288000 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.288082 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.288157 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 12:18:27.288253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.288501 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.288733 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.288976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.289214 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 12:18:27.289386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 12:18:27.318203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.318433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.318656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.318742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.318903 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.319134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.327377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.327850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.355008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.355238 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.360816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.361058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.411681 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.412579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.460811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.461039 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.462458 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.462702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.620693 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.620928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.621176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.621391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.621606 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.621820 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.623062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.623301 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.623527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.623774 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.623994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.624212 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.644736 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.644972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.645064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.645237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.645323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.645462 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.645547 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.645681 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.645776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.645903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.645992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.646121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.646206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.646366 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.657732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.658370 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.671079 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.671263 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.671528 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.671727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.671815 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.671966 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.672113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.672198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.672288 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.672394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.672479 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.672617 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.672836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.673050 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 12:18:27.689266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.689473 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 12:18:27.689698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.689817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.689903 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 12:18:27.690021 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 12:18:27.690240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.690324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.690439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 12:18:27.690545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.690718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.690984 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.691065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.691311 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.691497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.691790 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.701065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.701339 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.701425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.701611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.701695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.701894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.701979 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.702194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.708786 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.709138 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.712031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.712118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.712313 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.712396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.712595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.712678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.712878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.712963 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.713162 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.713244 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.713424 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.713514 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.713690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.713772 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.713963 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.714046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.714344 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.714427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.714621 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.714703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.715421 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.715514 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.738570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.740047 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.740337 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.743799 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.744967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.745256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.745341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.745539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.745623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.750273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.750569 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.750860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.752071 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.752364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.756444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.756741 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.757026 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.757215 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.757299 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.757511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.757594 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.757847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.758120 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.758396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.761636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.761939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.762230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.762523 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.762808 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.763093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.763372 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.763672 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.775875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.776161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.780645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.781379 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.781665 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.783906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.784207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.785543 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.788389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.788755 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.789060 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.791486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.793944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.795392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.795722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.796028 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.796334 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.796623 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.796913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.797096 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.800374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.800682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.800974 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.801286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.801574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.801881 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.812660 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.812962 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.813234 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.813505 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.817851 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.818146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.818417 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.818686 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.820417 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.821940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.822220 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.825676 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.825960 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.826254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.826508 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.826592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.826833 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.828065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.828349 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.830972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.831265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.831562 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.831881 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.832177 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.838103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.838390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.838674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.838961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.839246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.839531 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.840484 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.844163 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.844454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.844742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.845033 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.845322 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.845604 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.845884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.856223 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.859238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.861408 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.862139 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.863572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.864464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.865338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.865629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.865927 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.867342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.868811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.870282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.870571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.870870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.872415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.873891 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.878164 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.878164 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.878533 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.878640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.878640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.878828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.881792 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.882090 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.882381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.882669 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.885639 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.885934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.886219 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.891263 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.891560 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.891869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.893662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.893950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.894234 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.894512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.894800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.895077 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.895353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.895657 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.895933 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.896222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.896495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.896769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.897034 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.897296 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.898725 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 12:18:27.899015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.899302 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.899586 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.899890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.900189 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.900469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.900743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.901028 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.901319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.901607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.901875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.902154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.903003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 12:18:27.931942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561e0f6b39c0 pid: 81638 (default app) Aug 24 12:18:27.935371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170aa61-e837-47fa-a9f3-14c61ee3bf32 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:27.935680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170aa61-e837-47fa-a9f3-14c61ee3bf32 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:27.935957 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170aa61-e837-47fa-a9f3-14c61ee3bf32 None None] PATH_INFO: `/v3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:27.936951 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561e0f6b39c0 pid: 81637 (default app) Aug 24 12:18:27.940161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1/1] 10.222.0.58 () {50 vars in 791 bytes} [Wed Aug 24 12:18:27 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:34.616959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae56b4-f3d0-4d8d-a2e9-4d8632fbb5af None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:34.617476 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae56b4-f3d0-4d8d-a2e9-4d8632fbb5af None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:34.617566 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eae56b4-f3d0-4d8d-a2e9-4d8632fbb5af None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:34.621827 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1/2] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:18:34.628243 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943809a-0197-4764-b137-d51d144394dd None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:34.628656 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943809a-0197-4764-b137-d51d144394dd None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:34.628801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943809a-0197-4764-b137-d51d144394dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:34.700581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: INFO dbcounter [None req-1943809a-0197-4764-b137-d51d144394dd None None] Registered counter for database keystone Aug 24 12:18:34.719881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG oslo_db.sqlalchemy.engines [None req-1943809a-0197-4764-b137-d51d144394dd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81638) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 12:18:34.726003 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writer thread running {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 12:18:34.909789 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG passlib.handlers.bcrypt [None req-1943809a-0197-4764-b137-d51d144394dd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81638) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 12:18:34.910225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG passlib.handlers.bcrypt [None req-1943809a-0197-4764-b137-d51d144394dd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81638) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 12:18:34.964644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1943809a-0197-4764-b137-d51d144394dd None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:34.966388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1943809a-0197-4764-b137-d51d144394dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:35.094559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 2/3] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 12:18:35.552865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae56b4-f3d0-4d8d-a2e9-4d8632fbb5af None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:35.622306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: INFO dbcounter [None req-3eae56b4-f3d0-4d8d-a2e9-4d8632fbb5af None None] Registered counter for database keystone Aug 24 12:18:35.643694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG oslo_db.sqlalchemy.engines [None req-3eae56b4-f3d0-4d8d-a2e9-4d8632fbb5af None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81637) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 12:18:35.650268 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writer thread running {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 12:18:35.779514 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eae56b4-f3d0-4d8d-a2e9-4d8632fbb5af None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:35.783860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe6fe4-3b38-46da-ad78-d28962141ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:35.784484 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe6fe4-3b38-46da-ad78-d28962141ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:35.784768 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe6fe4-3b38-46da-ad78-d28962141ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:35.785043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe6fe4-3b38-46da-ad78-d28962141ff1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:35.793720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe6fe4-3b38-46da-ad78-d28962141ff1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:35.844880 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: WARNING keystone.common.rbac_enforcer.enforcer [None req-25fe6fe4-3b38-46da-ad78-d28962141ff1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 12:18:35.845177 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe6fe4-3b38-46da-ad78-d28962141ff1 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:35.854615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 2/4] 10.222.0.58 () {58 vars in 1324 bytes} [Wed Aug 24 12:18:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:18:35.861149 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943809a-0197-4764-b137-d51d144394dd None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:35.876462 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943809a-0197-4764-b137-d51d144394dd None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:35.881151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b60ef-4303-46b4-afd0-47242e77fe87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:35.882066 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b60ef-4303-46b4-afd0-47242e77fe87 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:35.882350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b60ef-4303-46b4-afd0-47242e77fe87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:35.882623 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b60ef-4303-46b4-afd0-47242e77fe87 admin admin] PATH_INFO: `/v3/projects/ae0194ad9d2f4f1ebdd6b4e39c18be2c` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:35.884228 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b60ef-4303-46b4-afd0-47242e77fe87 admin admin] RBAC: Authorizing `identity:get_project(project_id=ae0194ad9d2f4f1ebdd6b4e39c18be2c)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:35.937063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f29b60ef-4303-46b4-afd0-47242e77fe87 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 12:18:35.937407 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b60ef-4303-46b4-afd0-47242e77fe87 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:35.941966 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 3/5] 10.222.0.58 () {56 vars in 1207 bytes} [Wed Aug 24 12:18:35 2022] GET /identity/v3/projects/ae0194ad9d2f4f1ebdd6b4e39c18be2c => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:37.841990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b5c68-f144-41b4-a267-175efc289937 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:37.842597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b5c68-f144-41b4-a267-175efc289937 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:37.842597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b5c68-f144-41b4-a267-175efc289937 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:37.846750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 3/6] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:18:37.852262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf179b-17fa-46e3-bd75-7fdf2f09e4a8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:37.852703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf179b-17fa-46e3-bd75-7fdf2f09e4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:37.852821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf179b-17fa-46e3-bd75-7fdf2f09e4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:37.874091 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c4cf179b-17fa-46e3-bd75-7fdf2f09e4a8 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:37.875610 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c4cf179b-17fa-46e3-bd75-7fdf2f09e4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:37.883580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 4/7] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 12:18:38.153883 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b5c68-f144-41b4-a267-175efc289937 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:38.165878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b5c68-f144-41b4-a267-175efc289937 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:38.171176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee572c6f-efe5-4bdf-b1bb-f5c84880f978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:38.171862 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee572c6f-efe5-4bdf-b1bb-f5c84880f978 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:38.172169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee572c6f-efe5-4bdf-b1bb-f5c84880f978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:38.172446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee572c6f-efe5-4bdf-b1bb-f5c84880f978 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:38.180091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee572c6f-efe5-4bdf-b1bb-f5c84880f978 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:38.181860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee572c6f-efe5-4bdf-b1bb-f5c84880f978 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:38.190704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 4/8] 10.222.0.58 () {58 vars in 1324 bytes} [Wed Aug 24 12:18:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 12:18:38.196292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf179b-17fa-46e3-bd75-7fdf2f09e4a8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:38.205573 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf179b-17fa-46e3-bd75-7fdf2f09e4a8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:38.209910 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7de68-d9bb-4038-8af5-6faf6f6de4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:38.210535 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7de68-d9bb-4038-8af5-6faf6f6de4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:38.210817 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7de68-d9bb-4038-8af5-6faf6f6de4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:38.211091 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7de68-d9bb-4038-8af5-6faf6f6de4b9 admin admin] PATH_INFO: `/v3/users/ec5a3f02319e4ee5bdd1b0a6b24084c2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:38.212411 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7de68-d9bb-4038-8af5-6faf6f6de4b9 admin admin] RBAC: Authorizing `identity:get_user(user_id=ec5a3f02319e4ee5bdd1b0a6b24084c2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:38.214225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7de68-d9bb-4038-8af5-6faf6f6de4b9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:38.216286 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 5/9] 10.222.0.58 () {56 vars in 1198 bytes} [Wed Aug 24 12:18:38 2022] GET /identity/v3/users/ec5a3f02319e4ee5bdd1b0a6b24084c2 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:40.184167 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2b052-cbc6-4591-af4b-a683595d3087 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:40.184747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2b052-cbc6-4591-af4b-a683595d3087 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:40.184747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2b052-cbc6-4591-af4b-a683595d3087 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:40.185930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 5/10] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:18:40.191596 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d268e0-71ed-412f-aae9-d07803df5612 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:40.192059 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d268e0-71ed-412f-aae9-d07803df5612 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:40.192193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d268e0-71ed-412f-aae9-d07803df5612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:40.210146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-42d268e0-71ed-412f-aae9-d07803df5612 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:40.211320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-42d268e0-71ed-412f-aae9-d07803df5612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:40.228329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e1b51-57f3-4a30-8332-52938a1a6008 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:40.228622 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e1b51-57f3-4a30-8332-52938a1a6008 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:40.228903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e1b51-57f3-4a30-8332-52938a1a6008 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:40.230244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 6/11] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:18:40.236069 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5ebc3-2163-4258-95f0-4d1a911b3922 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:40.236357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5ebc3-2163-4258-95f0-4d1a911b3922 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:40.236632 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5ebc3-2163-4258-95f0-4d1a911b3922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:40.278270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG passlib.handlers.bcrypt [None req-b4e5ebc3-2163-4258-95f0-4d1a911b3922 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81637) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 12:18:40.278668 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG passlib.handlers.bcrypt [None req-b4e5ebc3-2163-4258-95f0-4d1a911b3922 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81637) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 12:18:40.291601 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 6/12] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:40.324795 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-b4e5ebc3-2163-4258-95f0-4d1a911b3922 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:40.326415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b4e5ebc3-2163-4258-95f0-4d1a911b3922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:40.423555 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 7/13] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:18:40.567959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d268e0-71ed-412f-aae9-d07803df5612 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:40.591928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:40.674635 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d268e0-71ed-412f-aae9-d07803df5612 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:40.691563 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5ebc3-2163-4258-95f0-4d1a911b3922 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:40.709531 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e567795-b055-4495-933a-c8bbb3f823d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:40.710173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e567795-b055-4495-933a-c8bbb3f823d7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:40.710454 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e567795-b055-4495-933a-c8bbb3f823d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:40.710729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e567795-b055-4495-933a-c8bbb3f823d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:40.722204 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e567795-b055-4495-933a-c8bbb3f823d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:18:40.731313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-5e567795-b055-4495-933a-c8bbb3f823d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:18:40.731313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:18:40.731313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:18:40.731313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:18:40.731313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:18:40.731313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:18:40.731313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:18:40.731313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:18:40.731313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:18:40.731313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:18:40.731313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:18:40.731313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:18:40.731313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:18:40.731313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:18:40.731313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:18:40.731313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:18:40.732457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:18:40.732457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:18:40.732457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:18:40.732457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:18:40.732457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:18:40.732457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:18:40.732457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:18:40.732457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:18:40.732457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:18:40.732457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:18:40.732457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:18:40.732457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:18:40.732457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:18:40.732457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:18:40.732457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:18:40.732457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:18:40.733343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:18:40.733343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:18:40.733343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:18:40.733343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:18:40.733343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:18:40.733343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:18:40.733343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:18:40.733343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:18:40.733343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:18:40.733343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:18:40.733343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:18:40.733343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:18:40.733343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 7/14] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:18:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:18:40.738333 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e567795-b055-4495-933a-c8bbb3f823d7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:40.796472 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5ebc3-2163-4258-95f0-4d1a911b3922 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:40.798737 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:40.819721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:40.835030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa57351-736b-4244-99f0-2e970e084066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:40.835919 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa57351-736b-4244-99f0-2e970e084066 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:40.836204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa57351-736b-4244-99f0-2e970e084066 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:40.836480 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa57351-736b-4244-99f0-2e970e084066 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:40.838179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e567795-b055-4495-933a-c8bbb3f823d7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:40.872861 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c1951-933e-4c4f-8706-9b9ba6ba9874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:40.873468 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c1951-933e-4c4f-8706-9b9ba6ba9874 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:40.874197 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c1951-933e-4c4f-8706-9b9ba6ba9874 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:40.874476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c1951-933e-4c4f-8706-9b9ba6ba9874 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:40.875149 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c1951-933e-4c4f-8706-9b9ba6ba9874 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:40.875470 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c1951-933e-4c4f-8706-9b9ba6ba9874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:40.876903 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c1951-933e-4c4f-8706-9b9ba6ba9874 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:40.889508 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 8/15] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:18:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:18:40.894941 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c1951-933e-4c4f-8706-9b9ba6ba9874 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:40.908034 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa57351-736b-4244-99f0-2e970e084066 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:40.909579 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa57351-736b-4244-99f0-2e970e084066 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:40.995204 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c1951-933e-4c4f-8706-9b9ba6ba9874 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:41.007822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 8/16] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:18:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:18:41.013479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa57351-736b-4244-99f0-2e970e084066 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:41.017147 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:41.030059 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6c4a8-c5ed-4d86-8784-d88cbe4bc8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:41.030689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6c4a8-c5ed-4d86-8784-d88cbe4bc8af None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:41.030970 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6c4a8-c5ed-4d86-8784-d88cbe4bc8af None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:41.031245 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6c4a8-c5ed-4d86-8784-d88cbe4bc8af None admin] PATH_INFO: `/v3/users/ec5a3f02319e4ee5bdd1b0a6b24084c2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:41.032391 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6c4a8-c5ed-4d86-8784-d88cbe4bc8af None admin] RBAC: Authorizing `identity:get_user(user_id=ec5a3f02319e4ee5bdd1b0a6b24084c2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:41.033871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6c4a8-c5ed-4d86-8784-d88cbe4bc8af None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:41.035463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 9/17] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:18:40 2022] GET /identity/v3/users/ec5a3f02319e4ee5bdd1b0a6b24084c2 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:18:41.040852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6c4a8-c5ed-4d86-8784-d88cbe4bc8af None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:41.057428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:41.114039 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa57351-736b-4244-99f0-2e970e084066 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:41.143774 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6c4a8-c5ed-4d86-8784-d88cbe4bc8af None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:41.147845 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6331ea-ab1b-4ed3-8bb0-fd212fbdc7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:41.148703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6331ea-ab1b-4ed3-8bb0-fd212fbdc7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:41.148987 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6331ea-ab1b-4ed3-8bb0-fd212fbdc7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:41.149265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6331ea-ab1b-4ed3-8bb0-fd212fbdc7e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:41.168635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6331ea-ab1b-4ed3-8bb0-fd212fbdc7e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:41.170415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6331ea-ab1b-4ed3-8bb0-fd212fbdc7e0 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:41.178378 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cac0c6-e672-4fbc-ad31-62ce6e1c82e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:41.179013 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cac0c6-e672-4fbc-ad31-62ce6e1c82e3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:41.179300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cac0c6-e672-4fbc-ad31-62ce6e1c82e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:41.179581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cac0c6-e672-4fbc-ad31-62ce6e1c82e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:41.180738 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cac0c6-e672-4fbc-ad31-62ce6e1c82e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:41.182180 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cac0c6-e672-4fbc-ad31-62ce6e1c82e3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:41.183912 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 10/18] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:18:41.188712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-8d6331ea-ab1b-4ed3-8bb0-fd212fbdc7e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:18:41.188712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:18:41.188712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:18:41.188712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:18:41.188712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:18:41.188712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:18:41.188712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:18:41.188712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:18:41.188712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:18:41.188712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:18:41.188712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:18:41.188712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:18:41.188712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:18:41.188712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:18:41.188712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:18:41.188712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:18:41.190085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:18:41.190893 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:18:41.190893 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:18:41.190893 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:18:41.191057 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cac0c6-e672-4fbc-ad31-62ce6e1c82e3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:41.191131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 9/19] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:18:41.195544 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6331ea-ab1b-4ed3-8bb0-fd212fbdc7e0 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:41.252962 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:41.270858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:41.290384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cac0c6-e672-4fbc-ad31-62ce6e1c82e3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:41.298170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6331ea-ab1b-4ed3-8bb0-fd212fbdc7e0 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:41.325070 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc11b3-d0b7-4163-9c28-2a7598a9b080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:41.325675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc11b3-d0b7-4163-9c28-2a7598a9b080 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:41.325956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc11b3-d0b7-4163-9c28-2a7598a9b080 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:41.326227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc11b3-d0b7-4163-9c28-2a7598a9b080 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:41.326945 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc11b3-d0b7-4163-9c28-2a7598a9b080 None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.domain.id=default, user.id=ec5a3f02319e4ee5bdd1b0a6b24084c2) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:41.327248 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc11b3-d0b7-4163-9c28-2a7598a9b080 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:41.328741 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc11b3-d0b7-4163-9c28-2a7598a9b080 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:41.332083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a37d0-9a3a-49f8-8001-b46a6b378ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:41.332701 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a37d0-9a3a-49f8-8001-b46a6b378ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:41.332991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a37d0-9a3a-49f8-8001-b46a6b378ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:41.333268 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a37d0-9a3a-49f8-8001-b46a6b378ff8 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:41.333914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a37d0-9a3a-49f8-8001-b46a6b378ff8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:41.334255 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a37d0-9a3a-49f8-8001-b46a6b378ff8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:41.335385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 11/20] 10.222.0.58 () {56 vars in 1322 bytes} [Wed Aug 24 12:18:41 2022] GET /identity/v3/role_assignments?user.id=ec5a3f02319e4ee5bdd1b0a6b24084c2&scope.domain.id=default&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:18:41.335746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a37d0-9a3a-49f8-8001-b46a6b378ff8 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:41.357708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 10/21] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:18:43.390104 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34782811-8b8d-4131-a8c1-8b301cfe3f93 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:43.391126 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34782811-8b8d-4131-a8c1-8b301cfe3f93 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:43.391655 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34782811-8b8d-4131-a8c1-8b301cfe3f93 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:43.393205 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 12/22] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:43.398578 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6f92c-df24-4d08-a0c8-9c97857181c8 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:43.399386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6f92c-df24-4d08-a0c8-9c97857181c8 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:43.399886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6f92c-df24-4d08-a0c8-9c97857181c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:43.425171 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-14d6f92c-df24-4d08-a0c8-9c97857181c8 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:43.426704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-14d6f92c-df24-4d08-a0c8-9c97857181c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:43.514353 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 11/23] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:43.823160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd982a22-fcf6-430f-a455-9b8a0e44d650 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:43.823546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd982a22-fcf6-430f-a455-9b8a0e44d650 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:43.823751 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd982a22-fcf6-430f-a455-9b8a0e44d650 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:43.826063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 13/24] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:43.831604 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbff8c3-0191-4ed8-9f3c-876cbe528328 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:43.832148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbff8c3-0191-4ed8-9f3c-876cbe528328 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:43.832639 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbff8c3-0191-4ed8-9f3c-876cbe528328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:43.856890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-4fbff8c3-0191-4ed8-9f3c-876cbe528328 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:43.858441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-4fbff8c3-0191-4ed8-9f3c-876cbe528328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:43.914646 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:43.944168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff916b59-d851-49ea-b5fe-ef3269fa77fe None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:43.944477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff916b59-d851-49ea-b5fe-ef3269fa77fe None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:43.944759 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff916b59-d851-49ea-b5fe-ef3269fa77fe None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:43.946289 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 14/25] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:18:43.951758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33756850-5d67-4916-9c04-a9b3b7c5f967 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:43.952046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33756850-5d67-4916-9c04-a9b3b7c5f967 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:43.952325 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33756850-5d67-4916-9c04-a9b3b7c5f967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:43.963020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 12/26] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:43.981452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-33756850-5d67-4916-9c04-a9b3b7c5f967 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:43.982675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-33756850-5d67-4916-9c04-a9b3b7c5f967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:44.074515 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 15/27] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:18:44.112944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbff8c3-0191-4ed8-9f3c-876cbe528328 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:44.251532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbff8c3-0191-4ed8-9f3c-876cbe528328 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:44.296902 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b646d0-788a-4777-b988-2d234b2de4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:44.297551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b646d0-788a-4777-b988-2d234b2de4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:44.297835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b646d0-788a-4777-b988-2d234b2de4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:44.298112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b646d0-788a-4777-b988-2d234b2de4b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:44.307340 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33756850-5d67-4916-9c04-a9b3b7c5f967 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:44.312445 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b646d0-788a-4777-b988-2d234b2de4b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:18:44.318868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-57b646d0-788a-4777-b988-2d234b2de4b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:18:44.318868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:18:44.318868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:18:44.318868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:18:44.318868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:18:44.318868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:18:44.318868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:18:44.318868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:18:44.318868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:18:44.318868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:18:44.318868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:18:44.318868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:18:44.318868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:18:44.318868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:18:44.318868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:18:44.318868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:18:44.319788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:18:44.319788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:18:44.319788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:18:44.319788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:18:44.319788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:18:44.319788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:18:44.319788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:18:44.319788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:18:44.319788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:18:44.319788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:18:44.319788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:18:44.319788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:18:44.319788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:18:44.319788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:18:44.319788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:18:44.319788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:18:44.320941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:18:44.320941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:18:44.320941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:18:44.320941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:18:44.320941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:18:44.320941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:18:44.320941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:18:44.320941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:18:44.320941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:18:44.320941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:18:44.320941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:18:44.320941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:18:44.320941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 13/28] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:18:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:18:44.326523 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b646d0-788a-4777-b988-2d234b2de4b0 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:44.396003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:44.456972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:44.486831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b646d0-788a-4777-b988-2d234b2de4b0 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:44.506165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33756850-5d67-4916-9c04-a9b3b7c5f967 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:44.526651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e818b0-0909-427e-94a7-297b56669346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:44.528261 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e818b0-0909-427e-94a7-297b56669346 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:44.528846 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e818b0-0909-427e-94a7-297b56669346 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:44.529318 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e818b0-0909-427e-94a7-297b56669346 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:44.530216 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e818b0-0909-427e-94a7-297b56669346 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:44.530724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e818b0-0909-427e-94a7-297b56669346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:44.532802 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e818b0-0909-427e-94a7-297b56669346 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:44.546064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 14/29] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:18:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:18:44.547738 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c7c55-c8db-4c6b-a7ce-6fac68dd9cd8 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:44.548225 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c7c55-c8db-4c6b-a7ce-6fac68dd9cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:44.548698 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c7c55-c8db-4c6b-a7ce-6fac68dd9cd8 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:44.550298 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 15/30] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:44.552853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c7c55-c8db-4c6b-a7ce-6fac68dd9cd8 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:44.576921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca05f02-c965-4466-9068-37797557ef08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:44.577568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca05f02-c965-4466-9068-37797557ef08 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:44.577852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca05f02-c965-4466-9068-37797557ef08 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:44.578129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca05f02-c965-4466-9068-37797557ef08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:44.592744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca05f02-c965-4466-9068-37797557ef08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:44.594268 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca05f02-c965-4466-9068-37797557ef08 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:44.625241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-4ca05f02-c965-4466-9068-37797557ef08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:18:44.625241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:18:44.625241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:18:44.625241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:18:44.625241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:18:44.625241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:18:44.625241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:18:44.625241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:18:44.625241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:18:44.625241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:18:44.625241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:18:44.625241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:18:44.625241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:18:44.625241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:18:44.625241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:18:44.625241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:18:44.626187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:18:44.627086 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:18:44.627086 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:18:44.627086 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:18:44.627086 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 16/31] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:18:44.632341 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939001c0-a570-4814-b4fc-97393d4fde04 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:44.632629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939001c0-a570-4814-b4fc-97393d4fde04 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:44.632907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939001c0-a570-4814-b4fc-97393d4fde04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:44.653906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-939001c0-a570-4814-b4fc-97393d4fde04 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:44.661135 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-939001c0-a570-4814-b4fc-97393d4fde04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:44.676716 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:44.682080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c7c55-c8db-4c6b-a7ce-6fac68dd9cd8 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:44.721053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78ee2e-23bb-4c5a-a4c3-ed00f3135eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:44.722108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78ee2e-23bb-4c5a-a4c3-ed00f3135eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:44.722630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78ee2e-23bb-4c5a-a4c3-ed00f3135eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:44.723181 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78ee2e-23bb-4c5a-a4c3-ed00f3135eb0 None admin] PATH_INFO: `/v3/users/ec5a3f02319e4ee5bdd1b0a6b24084c2` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:44.725623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78ee2e-23bb-4c5a-a4c3-ed00f3135eb0 None admin] RBAC: Authorizing `identity:get_user(user_id=ec5a3f02319e4ee5bdd1b0a6b24084c2)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:44.728330 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78ee2e-23bb-4c5a-a4c3-ed00f3135eb0 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:44.730190 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 16/32] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:18:44 2022] GET /identity/v3/users/ec5a3f02319e4ee5bdd1b0a6b24084c2 => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:18:44.731191 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78ee2e-23bb-4c5a-a4c3-ed00f3135eb0 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:44.732560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 17/33] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:44.740541 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939001c0-a570-4814-b4fc-97393d4fde04 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:44.847274 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:44.897654 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78ee2e-23bb-4c5a-a4c3-ed00f3135eb0 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:44.902008 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939001c0-a570-4814-b4fc-97393d4fde04 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:44.942558 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dd9a5-3dc3-4391-ba15-7c0c90a5d438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:44.943548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dd9a5-3dc3-4391-ba15-7c0c90a5d438 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:44.944064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dd9a5-3dc3-4391-ba15-7c0c90a5d438 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:44.944562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dd9a5-3dc3-4391-ba15-7c0c90a5d438 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:44.945665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24dd9a5-3dc3-4391-ba15-7c0c90a5d438 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:44.946510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24dd9a5-3dc3-4391-ba15-7c0c90a5d438 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:44.948311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24dd9a5-3dc3-4391-ba15-7c0c90a5d438 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:44.970459 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 18/34] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:44.971709 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dd9a5-3dc3-4391-ba15-7c0c90a5d438 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:45.005769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d2d70-92e8-4342-b808-71839edf16f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:45.006397 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d2d70-92e8-4342-b808-71839edf16f9 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:45.006682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d2d70-92e8-4342-b808-71839edf16f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:45.006966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d2d70-92e8-4342-b808-71839edf16f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:45.007664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d2d70-92e8-4342-b808-71839edf16f9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:45.009053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79d2d70-92e8-4342-b808-71839edf16f9 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:45.083216 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dd9a5-3dc3-4391-ba15-7c0c90a5d438 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:45.102063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:45.122793 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb304d-2d49-425a-8e90-7df9cf0aaf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:45.123441 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb304d-2d49-425a-8e90-7df9cf0aaf7e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:45.123751 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb304d-2d49-425a-8e90-7df9cf0aaf7e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:45.124029 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb304d-2d49-425a-8e90-7df9cf0aaf7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:45.125202 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb304d-2d49-425a-8e90-7df9cf0aaf7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:45.126870 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb304d-2d49-425a-8e90-7df9cf0aaf7e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:45.133339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 19/35] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:45.133339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb304d-2d49-425a-8e90-7df9cf0aaf7e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:45.149475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 17/36] 10.222.0.58 () {60 vars in 1131 bytes} [Wed Aug 24 12:18:44 2022] POST /identity/v3/roles => generated 219 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:18:45.149475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b22b2-4deb-4cc0-a8b3-28a76cf47e1d None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:45.149475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b22b2-4deb-4cc0-a8b3-28a76cf47e1d None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:45.149475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b22b2-4deb-4cc0-a8b3-28a76cf47e1d None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:45.149475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 18/37] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:45.149475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b22b2-4deb-4cc0-a8b3-28a76cf47e1d None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:45.182434 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:45.260567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb304d-2d49-425a-8e90-7df9cf0aaf7e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:45.299275 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b22b2-4deb-4cc0-a8b3-28a76cf47e1d None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:45.300491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87394b67-6892-4d5d-ab56-4e50f67a83a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:45.302043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87394b67-6892-4d5d-ab56-4e50f67a83a8 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:45.302043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87394b67-6892-4d5d-ab56-4e50f67a83a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:45.302043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87394b67-6892-4d5d-ab56-4e50f67a83a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:45.302385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87394b67-6892-4d5d-ab56-4e50f67a83a8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:45.303921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87394b67-6892-4d5d-ab56-4e50f67a83a8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:45.342603 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3e5c7-5f7d-4997-bf35-2de2360be450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:45.343272 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3e5c7-5f7d-4997-bf35-2de2360be450 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:45.343578 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3e5c7-5f7d-4997-bf35-2de2360be450 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:45.343910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3e5c7-5f7d-4997-bf35-2de2360be450 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:45.344580 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3e5c7-5f7d-4997-bf35-2de2360be450 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:45.346084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3e5c7-5f7d-4997-bf35-2de2360be450 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:45.363053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 19/38] 10.222.0.58 () {60 vars in 1131 bytes} [Wed Aug 24 12:18:45 2022] POST /identity/v3/roles => generated 225 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:18:45.363655 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3e5c7-5f7d-4997-bf35-2de2360be450 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:45.450512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 20/39] 10.222.0.58 () {60 vars in 1141 bytes} [Wed Aug 24 12:18:45 2022] POST /identity/v3/projects => generated 301 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:18:45.452190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681dd80f-7667-4cde-aba2-c7013f05ef26 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:45.452491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681dd80f-7667-4cde-aba2-c7013f05ef26 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:45.452780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681dd80f-7667-4cde-aba2-c7013f05ef26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:45.474080 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-681dd80f-7667-4cde-aba2-c7013f05ef26 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:45.475754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-681dd80f-7667-4cde-aba2-c7013f05ef26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:45.476505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:45.496404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3e5c7-5f7d-4997-bf35-2de2360be450 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:45.518281 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:45.552237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37e033-aed4-449b-83e3-d6d08c45c2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:45.553006 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37e033-aed4-449b-83e3-d6d08c45c2c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:45.553297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37e033-aed4-449b-83e3-d6d08c45c2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:45.553586 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d37e033-aed4-449b-83e3-d6d08c45c2c3 None admin] PATH_INFO: `/v3/domains/default/users/ec5a3f02319e4ee5bdd1b0a6b24084c2/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:45.555871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37e033-aed4-449b-83e3-d6d08c45c2c3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ec5a3f02319e4ee5bdd1b0a6b24084c2, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:45.558280 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d37e033-aed4-449b-83e3-d6d08c45c2c3 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:45.576913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 21/40] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:45.579401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd527e3-1467-4be0-a6c7-cbf87d6b0bff None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:45.579716 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd527e3-1467-4be0-a6c7-cbf87d6b0bff None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:45.580001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd527e3-1467-4be0-a6c7-cbf87d6b0bff None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:45.581352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 22/41] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:45.583095 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 20/42] 10.222.0.58 () {58 vars in 1361 bytes} [Wed Aug 24 12:18:45 2022] PUT /identity/v3/domains/default/users/ec5a3f02319e4ee5bdd1b0a6b24084c2/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:18:45.587582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e807314-e1bb-4791-a782-bdc5eca1d0fc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:45.587905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e807314-e1bb-4791-a782-bdc5eca1d0fc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:45.588189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e807314-e1bb-4791-a782-bdc5eca1d0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:45.610438 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3e807314-e1bb-4791-a782-bdc5eca1d0fc None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:45.611798 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3e807314-e1bb-4791-a782-bdc5eca1d0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:45.695421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 23/43] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:45.871291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37e033-aed4-449b-83e3-d6d08c45c2c3 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:45.967949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e807314-e1bb-4791-a782-bdc5eca1d0fc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:45.978064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d37e033-aed4-449b-83e3-d6d08c45c2c3 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:46.015183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44604a7-feca-4329-b08e-57da0a6d5475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:46.015951 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44604a7-feca-4329-b08e-57da0a6d5475 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:46.016238 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44604a7-feca-4329-b08e-57da0a6d5475 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:46.016524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44604a7-feca-4329-b08e-57da0a6d5475 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:46.017195 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44604a7-feca-4329-b08e-57da0a6d5475 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:46.018718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44604a7-feca-4329-b08e-57da0a6d5475 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:46.035659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 21/44] 10.222.0.58 () {60 vars in 1131 bytes} [Wed Aug 24 12:18:45 2022] POST /identity/v3/roles => generated 223 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:18:46.074832 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:46.082895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e807314-e1bb-4791-a782-bdc5eca1d0fc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:46.233428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7be32-7a10-4192-86d3-9d5364b86dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:46.234116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7be32-7a10-4192-86d3-9d5364b86dcf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:46.234410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7be32-7a10-4192-86d3-9d5364b86dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:46.234691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7be32-7a10-4192-86d3-9d5364b86dcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:46.235846 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7be32-7a10-4192-86d3-9d5364b86dcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:46.237451 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7be32-7a10-4192-86d3-9d5364b86dcf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:46.239054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 24/45] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:18:46.244712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44604a7-feca-4329-b08e-57da0a6d5475 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:46.254480 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:46.345925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44604a7-feca-4329-b08e-57da0a6d5475 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:46.381801 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b014c92-4952-4453-af28-6031d3f74844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:46.382423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b014c92-4952-4453-af28-6031d3f74844 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:46.382710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b014c92-4952-4453-af28-6031d3f74844 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:46.382991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b014c92-4952-4453-af28-6031d3f74844 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:46.383671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b014c92-4952-4453-af28-6031d3f74844 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:46.385139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b014c92-4952-4453-af28-6031d3f74844 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:46.414919 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 22/46] 10.222.0.58 () {60 vars in 1141 bytes} [Wed Aug 24 12:18:46 2022] POST /identity/v3/projects => generated 312 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:18:47.536303 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e319b-ef52-43e6-a7f7-e4bccdb8a825 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:47.536819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e319b-ef52-43e6-a7f7-e4bccdb8a825 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:47.536921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e319b-ef52-43e6-a7f7-e4bccdb8a825 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:47.538209 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 25/47] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:47.544290 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a670e-d64e-442b-826e-9b3d2baac9cb None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:47.544723 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a670e-d64e-442b-826e-9b3d2baac9cb None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:47.544921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a670e-d64e-442b-826e-9b3d2baac9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:47.563511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-ad0a670e-d64e-442b-826e-9b3d2baac9cb None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:47.565020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-ad0a670e-d64e-442b-826e-9b3d2baac9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:47.616119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:47.631114 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 23/48] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:47.906618 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e319b-ef52-43e6-a7f7-e4bccdb8a825 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:48.007144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e319b-ef52-43e6-a7f7-e4bccdb8a825 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:48.040239 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f0461-90c2-4ff7-9e19-a51dc411e2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:48.040877 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f0461-90c2-4ff7-9e19-a51dc411e2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:48.041160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f0461-90c2-4ff7-9e19-a51dc411e2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:48.041434 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f0461-90c2-4ff7-9e19-a51dc411e2c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:48.052346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f0461-90c2-4ff7-9e19-a51dc411e2c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:18:48.057130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-7b4f0461-90c2-4ff7-9e19-a51dc411e2c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:18:48.057130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:18:48.057130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:18:48.057130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:18:48.057130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:18:48.057130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:18:48.057130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:18:48.057130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:18:48.057130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:18:48.057130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:18:48.057130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:18:48.057130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:18:48.057130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:18:48.057130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:18:48.057130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:18:48.057130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:18:48.057947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:18:48.057947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:18:48.057947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:18:48.057947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:18:48.057947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:18:48.057947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:18:48.057947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:18:48.057947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:18:48.057947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:18:48.057947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:18:48.057947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:18:48.057947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:18:48.057947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:18:48.057947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:18:48.057947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:18:48.057947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:18:48.059932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:18:48.059932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:18:48.059932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:18:48.059932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:18:48.059932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:18:48.059932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:18:48.059932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:18:48.059932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:18:48.059932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:18:48.059932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:18:48.059932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:18:48.059932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:18:48.059932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 26/49] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:18:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:18:48.064713 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a670e-d64e-442b-826e-9b3d2baac9cb None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:48.163487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a670e-d64e-442b-826e-9b3d2baac9cb None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:48.198728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879ae47-6f0d-4bbb-81c1-898f01ffd2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:48.199688 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879ae47-6f0d-4bbb-81c1-898f01ffd2ce None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:48.199918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879ae47-6f0d-4bbb-81c1-898f01ffd2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:48.200193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879ae47-6f0d-4bbb-81c1-898f01ffd2ce None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:48.200859 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879ae47-6f0d-4bbb-81c1-898f01ffd2ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:48.201169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879ae47-6f0d-4bbb-81c1-898f01ffd2ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:48.202552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879ae47-6f0d-4bbb-81c1-898f01ffd2ce None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:48.210938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 24/50] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:18:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:48.216246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f0461-90c2-4ff7-9e19-a51dc411e2c9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:48.241426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:48.314220 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f0461-90c2-4ff7-9e19-a51dc411e2c9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:48.346341 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3683460-ef0c-4ab4-a919-000dd40ffcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:48.346785 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726eb95-dfca-4481-b41a-5471dcb3e92f None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:48.346988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3683460-ef0c-4ab4-a919-000dd40ffcc5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:48.347077 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726eb95-dfca-4481-b41a-5471dcb3e92f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:48.347269 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3683460-ef0c-4ab4-a919-000dd40ffcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:48.347355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726eb95-dfca-4481-b41a-5471dcb3e92f None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:48.347541 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3683460-ef0c-4ab4-a919-000dd40ffcc5 None admin] PATH_INFO: `/v3/users/ec5a3f02319e4ee5bdd1b0a6b24084c2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:48.348680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3683460-ef0c-4ab4-a919-000dd40ffcc5 None admin] RBAC: Authorizing `identity:get_user(user_id=ec5a3f02319e4ee5bdd1b0a6b24084c2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:48.348773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 25/51] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:18:48.350140 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3683460-ef0c-4ab4-a919-000dd40ffcc5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:48.351851 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 27/52] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:18:48 2022] GET /identity/v3/users/ec5a3f02319e4ee5bdd1b0a6b24084c2 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:48.353742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca57be-9ff2-4afa-9630-6c7c55774370 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:48.354036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca57be-9ff2-4afa-9630-6c7c55774370 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:48.354322 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca57be-9ff2-4afa-9630-6c7c55774370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:48.357691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3683460-ef0c-4ab4-a919-000dd40ffcc5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:48.373093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-0aca57be-9ff2-4afa-9630-6c7c55774370 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:48.374300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-0aca57be-9ff2-4afa-9630-6c7c55774370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:48.421595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:48.442573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 26/53] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:48.451229 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:48.459905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3683460-ef0c-4ab4-a919-000dd40ffcc5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:48.500147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4be0cf-975f-421b-8c20-66fc010bc65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:48.500782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4be0cf-975f-421b-8c20-66fc010bc65f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:48.501064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4be0cf-975f-421b-8c20-66fc010bc65f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:48.501338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4be0cf-975f-421b-8c20-66fc010bc65f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:48.502448 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4be0cf-975f-421b-8c20-66fc010bc65f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:48.503935 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4be0cf-975f-421b-8c20-66fc010bc65f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:48.505526 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 28/54] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:18:48.511424 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca57be-9ff2-4afa-9630-6c7c55774370 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:48.712747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca57be-9ff2-4afa-9630-6c7c55774370 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:48.717907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4be0cf-975f-421b-8c20-66fc010bc65f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:48.748145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59145d30-0d15-402c-abf4-2508321660fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:48.748760 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59145d30-0d15-402c-abf4-2508321660fa None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:48.749041 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59145d30-0d15-402c-abf4-2508321660fa None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:48.749317 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59145d30-0d15-402c-abf4-2508321660fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:48.750040 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59145d30-0d15-402c-abf4-2508321660fa None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.domain.id=default, user.id=ec5a3f02319e4ee5bdd1b0a6b24084c2) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:48.750348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59145d30-0d15-402c-abf4-2508321660fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:48.751809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59145d30-0d15-402c-abf4-2508321660fa None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:48.759466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 27/55] 10.222.0.58 () {56 vars in 1322 bytes} [Wed Aug 24 12:18:48 2022] GET /identity/v3/role_assignments?user.id=ec5a3f02319e4ee5bdd1b0a6b24084c2&scope.domain.id=default&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 530 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:48.826303 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4be0cf-975f-421b-8c20-66fc010bc65f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:48.862030 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44e23b-1944-4d09-8a61-2fb8ab4be113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:48.862682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44e23b-1944-4d09-8a61-2fb8ab4be113 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:48.862964 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44e23b-1944-4d09-8a61-2fb8ab4be113 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:48.863247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44e23b-1944-4d09-8a61-2fb8ab4be113 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:48.864401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44e23b-1944-4d09-8a61-2fb8ab4be113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:48.865881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44e23b-1944-4d09-8a61-2fb8ab4be113 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:48.867506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 29/56] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:18:48.873380 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59145d30-0d15-402c-abf4-2508321660fa None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:48.921968 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:48.978369 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59145d30-0d15-402c-abf4-2508321660fa None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:49.010762 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d3829-1dd3-4737-b92e-cfcb5e8a20a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:49.011367 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d3829-1dd3-4737-b92e-cfcb5e8a20a8 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:49.012207 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d3829-1dd3-4737-b92e-cfcb5e8a20a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:49.012483 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d3829-1dd3-4737-b92e-cfcb5e8a20a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:49.013200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d3829-1dd3-4737-b92e-cfcb5e8a20a8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:49.014669 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d3829-1dd3-4737-b92e-cfcb5e8a20a8 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:49.039701 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 28/57] 10.222.0.58 () {60 vars in 1140 bytes} [Wed Aug 24 12:18:48 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:18:50.951013 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fa7b7-7930-4fa3-a6e6-5c7b64d2e323 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:50.951525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fa7b7-7930-4fa3-a6e6-5c7b64d2e323 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:50.951613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fa7b7-7930-4fa3-a6e6-5c7b64d2e323 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:50.952972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 30/58] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:50.958060 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97666ad-786c-4b26-ae8d-86185359045c None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:50.958434 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97666ad-786c-4b26-ae8d-86185359045c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:50.958613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97666ad-786c-4b26-ae8d-86185359045c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:50.976832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-f97666ad-786c-4b26-ae8d-86185359045c None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:50.978031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f97666ad-786c-4b26-ae8d-86185359045c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:51.043602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 29/59] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:18:51.315336 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fa7b7-7930-4fa3-a6e6-5c7b64d2e323 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:51.343410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:51.415279 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fa7b7-7930-4fa3-a6e6-5c7b64d2e323 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:51.448252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d8d64-5d1b-47db-80c9-2d16d34f1fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:51.448880 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d8d64-5d1b-47db-80c9-2d16d34f1fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:51.449494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d8d64-5d1b-47db-80c9-2d16d34f1fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:51.449993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d8d64-5d1b-47db-80c9-2d16d34f1fd1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:51.451199 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d8d64-5d1b-47db-80c9-2d16d34f1fd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:51.452691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d8d64-5d1b-47db-80c9-2d16d34f1fd1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:51.454281 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 31/60] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:51.460125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97666ad-786c-4b26-ae8d-86185359045c None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:51.487487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:51.558303 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97666ad-786c-4b26-ae8d-86185359045c None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:51.591462 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c106dad-c1a6-471d-b164-91561fdb59db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:51.592048 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c106dad-c1a6-471d-b164-91561fdb59db None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:51.592335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c106dad-c1a6-471d-b164-91561fdb59db None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:51.592612 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c106dad-c1a6-471d-b164-91561fdb59db None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:51.593268 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c106dad-c1a6-471d-b164-91561fdb59db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:51.594737 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c106dad-c1a6-471d-b164-91561fdb59db None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:51.621330 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 30/61] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:18:51 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:18:54.537957 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8746e-4000-4e42-b82d-47aaa24920b9 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:54.538968 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8746e-4000-4e42-b82d-47aaa24920b9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:54.539465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8746e-4000-4e42-b82d-47aaa24920b9 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:54.541070 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 32/62] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:54.547113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9652cf-a57c-419a-a508-e226ff7cde3f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:54.547944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9652cf-a57c-419a-a508-e226ff7cde3f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:54.548438 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9652cf-a57c-419a-a508-e226ff7cde3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:54.571656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-1f9652cf-a57c-419a-a508-e226ff7cde3f None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:54.573165 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1f9652cf-a57c-419a-a508-e226ff7cde3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:54.674343 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 31/63] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:18:54.694146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e96658-7b6f-4ca5-847c-2dea980a8766 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:54.694146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e96658-7b6f-4ca5-847c-2dea980a8766 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:54.694146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e96658-7b6f-4ca5-847c-2dea980a8766 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:54.694900 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 33/64] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:54.701630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de4766-820b-43e3-9e8b-17e53f1c0e69 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:54.701999 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de4766-820b-43e3-9e8b-17e53f1c0e69 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:54.702282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de4766-820b-43e3-9e8b-17e53f1c0e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:54.725461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-a9de4766-820b-43e3-9e8b-17e53f1c0e69 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:54.726981 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a9de4766-820b-43e3-9e8b-17e53f1c0e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:54.735141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:54.829740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 32/65] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:54.962864 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e96658-7b6f-4ca5-847c-2dea980a8766 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:55.094126 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:55.104287 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e96658-7b6f-4ca5-847c-2dea980a8766 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:55.154699 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e68b06-455b-4778-a864-d0e98901c901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:55.155343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e68b06-455b-4778-a864-d0e98901c901 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:55.155649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e68b06-455b-4778-a864-d0e98901c901 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:55.155926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e68b06-455b-4778-a864-d0e98901c901 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:55.164692 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de4766-820b-43e3-9e8b-17e53f1c0e69 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:55.179670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e68b06-455b-4778-a864-d0e98901c901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:18:55.179670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-54e68b06-455b-4778-a864-d0e98901c901 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:18:55.179670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:18:55.179670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:18:55.179670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:18:55.179670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:18:55.179670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:18:55.179670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:18:55.179670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:18:55.179670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:18:55.179670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:18:55.179670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:18:55.179670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:18:55.179670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:18:55.181812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:18:55.183162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:18:55.183162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:18:55.183162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:18:55.183162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:18:55.183162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:18:55.183162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:18:55.183162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:18:55.183162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:18:55.183162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:18:55.183162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:18:55.183162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:18:55.183162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:18:55.183162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:18:55.183162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:18:55.183162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 34/66] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:18:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:18:55.187367 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e68b06-455b-4778-a864-d0e98901c901 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:55.309166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e68b06-455b-4778-a864-d0e98901c901 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:55.312454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de4766-820b-43e3-9e8b-17e53f1c0e69 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:55.343318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bda37-e265-46c0-a051-7940ea8ef728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:55.343965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bda37-e265-46c0-a051-7940ea8ef728 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:55.344306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bda37-e265-46c0-a051-7940ea8ef728 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:55.344543 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bda37-e265-46c0-a051-7940ea8ef728 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:55.345205 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bda37-e265-46c0-a051-7940ea8ef728 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:55.345515 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bda37-e265-46c0-a051-7940ea8ef728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:55.346262 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af16d0d9-fbcb-4902-b69d-0882c0851537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:55.346896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16d0d9-fbcb-4902-b69d-0882c0851537 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:55.346979 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bda37-e265-46c0-a051-7940ea8ef728 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:55.347182 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16d0d9-fbcb-4902-b69d-0882c0851537 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:55.347459 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16d0d9-fbcb-4902-b69d-0882c0851537 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:55.360124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 35/67] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:18:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:55.360124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cf5e7-2468-4289-ad22-5eeb31092403 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:55.360124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cf5e7-2468-4289-ad22-5eeb31092403 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:55.360124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cf5e7-2468-4289-ad22-5eeb31092403 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:55.360467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af16d0d9-fbcb-4902-b69d-0882c0851537 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:18:55.360547 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 36/68] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:55.360547 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0386fbb-2f8c-4211-9487-0f10b72c42cc None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:55.360547 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0386fbb-2f8c-4211-9487-0f10b72c42cc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:55.360547 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0386fbb-2f8c-4211-9487-0f10b72c42cc None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:55.361896 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 37/69] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:55.363034 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e8a79-6bde-4666-8c7e-ee5692722adc None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:55.363327 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e8a79-6bde-4666-8c7e-ee5692722adc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:55.363607 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e8a79-6bde-4666-8c7e-ee5692722adc None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:55.364135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-af16d0d9-fbcb-4902-b69d-0882c0851537 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:18:55.364135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:18:55.364135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:18:55.364135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:18:55.364135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:18:55.364135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:18:55.364135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:18:55.364135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:18:55.364135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:18:55.364135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:18:55.364135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:18:55.364135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:18:55.364135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:18:55.364135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:18:55.364135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:18:55.364135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:18:55.364969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:18:55.364969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:18:55.364969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:18:55.364969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:18:55.364969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:18:55.364969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:18:55.364969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:18:55.364969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:18:55.364969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:18:55.364969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:18:55.364969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:18:55.364969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:18:55.364969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:18:55.364969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:18:55.364969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:18:55.364969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:18:55.365729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:18:55.365729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:18:55.365729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:18:55.365729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:18:55.365729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:18:55.365729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:18:55.365729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:18:55.365729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:18:55.365729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:18:55.365729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:18:55.365729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:18:55.365729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:18:55.366306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 38/70] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:55.366399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 33/71] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:18:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:18:55.366467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e8a79-6bde-4666-8c7e-ee5692722adc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:55.371319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e45861-4ff1-4aa0-89e1-4d68f607fd7e None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:55.371692 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e45861-4ff1-4aa0-89e1-4d68f607fd7e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:55.371959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e45861-4ff1-4aa0-89e1-4d68f607fd7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:55.378186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:55.385943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:55.391395 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-43e45861-4ff1-4aa0-89e1-4d68f607fd7e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:55.392638 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-43e45861-4ff1-4aa0-89e1-4d68f607fd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:55.459446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 34/72] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:55.460645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eddff-c9a7-4c61-9f31-e473c079087f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:55.461002 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eddff-c9a7-4c61-9f31-e473c079087f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:55.461368 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eddff-c9a7-4c61-9f31-e473c079087f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:55.478873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-9e1eddff-c9a7-4c61-9f31-e473c079087f None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:55.480109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-9e1eddff-c9a7-4c61-9f31-e473c079087f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:55.490726 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e8a79-6bde-4666-8c7e-ee5692722adc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:55.525976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa414a6-6e33-4c4f-9fc6-034258643996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:55.526836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa414a6-6e33-4c4f-9fc6-034258643996 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:55.527129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa414a6-6e33-4c4f-9fc6-034258643996 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:55.528809 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa414a6-6e33-4c4f-9fc6-034258643996 None admin] PATH_INFO: `/v3/users/6c5badd8f28a4d749446523e9d5a6ed3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:55.544505 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa414a6-6e33-4c4f-9fc6-034258643996 None admin] RBAC: Authorizing `identity:get_user(user_id=6c5badd8f28a4d749446523e9d5a6ed3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:55.546284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa414a6-6e33-4c4f-9fc6-034258643996 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:55.548074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 39/73] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:18:55 2022] GET /identity/v3/users/6c5badd8f28a4d749446523e9d5a6ed3 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:55.548849 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa414a6-6e33-4c4f-9fc6-034258643996 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:55.558877 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 35/74] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:55.560865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a279e-642a-4261-854f-47129d7c384a None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:55.561346 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a279e-642a-4261-854f-47129d7c384a None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:55.561782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a279e-642a-4261-854f-47129d7c384a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:55.580201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-b91a279e-642a-4261-854f-47129d7c384a None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:55.581843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b91a279e-642a-4261-854f-47129d7c384a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:55.644667 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:55.657639 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 36/75] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:55.658402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a279e-642a-4261-854f-47129d7c384a None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:55.660618 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:55.700228 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa414a6-6e33-4c4f-9fc6-034258643996 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:55.762195 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a279e-642a-4261-854f-47129d7c384a None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:55.762685 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710adb5-997a-4e64-8d09-8db7be6925a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:55.763389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710adb5-997a-4e64-8d09-8db7be6925a4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:55.763972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710adb5-997a-4e64-8d09-8db7be6925a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:55.764257 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710adb5-997a-4e64-8d09-8db7be6925a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:55.765825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710adb5-997a-4e64-8d09-8db7be6925a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:55.766141 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710adb5-997a-4e64-8d09-8db7be6925a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:55.767607 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710adb5-997a-4e64-8d09-8db7be6925a4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:55.778942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 40/76] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:18:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:55.780581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710adb5-997a-4e64-8d09-8db7be6925a4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:55.802221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3b452-4f76-4b55-883e-8d1890473a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:55.803009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3b452-4f76-4b55-883e-8d1890473a78 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:55.803390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3b452-4f76-4b55-883e-8d1890473a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:55.803766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a3b452-4f76-4b55-883e-8d1890473a78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:55.869765 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:55.884434 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3b452-4f76-4b55-883e-8d1890473a78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:55.886382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a3b452-4f76-4b55-883e-8d1890473a78 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:55.906481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710adb5-997a-4e64-8d09-8db7be6925a4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:55.944114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d0a90-b03e-4e28-9202-338e18c672e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:55.944849 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d0a90-b03e-4e28-9202-338e18c672e1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:55.945142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d0a90-b03e-4e28-9202-338e18c672e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:55.945705 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d0a90-b03e-4e28-9202-338e18c672e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:55.954147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:55.959401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63d0a90-b03e-4e28-9202-338e18c672e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:18:55.964693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d63d0a90-b03e-4e28-9202-338e18c672e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:18:55.964693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:18:55.964693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:18:55.964693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:18:55.964693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:18:55.964693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:18:55.964693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:18:55.964693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:18:55.964693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:18:55.964693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:18:55.964693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:18:55.964693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:18:55.964693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:18:55.964693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:18:55.964693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:18:55.964693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:18:55.965722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:18:55.965722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:18:55.965722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:18:55.965722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:18:55.965722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:18:55.965722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:18:55.965722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:18:55.965722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:18:55.965722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:18:55.965722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:18:55.965722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:18:55.965722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:18:55.965722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:18:55.965722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:18:55.965722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:18:55.965722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:18:55.967054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:18:55.967054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:18:55.967054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:18:55.967054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:18:55.967054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:18:55.967054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:18:55.967054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:18:55.967054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:18:55.967054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:18:55.967054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:18:55.967054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:18:55.967054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:18:55.967054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 41/77] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:18:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:18:55.968495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d0a90-b03e-4e28-9202-338e18c672e1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:55.996417 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 37/78] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:18:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:18:55.998080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3b452-4f76-4b55-883e-8d1890473a78 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.072824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d0a90-b03e-4e28-9202-338e18c672e1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:56.090820 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:56.099587 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a3b452-4f76-4b55-883e-8d1890473a78 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:56.106944 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc6fe4-3b96-416b-a82a-0e0a758353ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:56.107582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc6fe4-3b96-416b-a82a-0e0a758353ee None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.107967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc6fe4-3b96-416b-a82a-0e0a758353ee None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.108249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc6fe4-3b96-416b-a82a-0e0a758353ee None admin] PATH_INFO: `/v3/users/ec5a3f02319e4ee5bdd1b0a6b24084c2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.109399 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc6fe4-3b96-416b-a82a-0e0a758353ee None admin] RBAC: Authorizing `identity:get_user(user_id=ec5a3f02319e4ee5bdd1b0a6b24084c2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:56.110888 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc6fe4-3b96-416b-a82a-0e0a758353ee None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:56.112512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 42/79] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:18:55 2022] GET /identity/v3/users/ec5a3f02319e4ee5bdd1b0a6b24084c2 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:56.113363 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc6fe4-3b96-416b-a82a-0e0a758353ee None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.138490 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f15e0d-527e-48d4-9638-acf7273aa57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:56.139125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f15e0d-527e-48d4-9638-acf7273aa57c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.139408 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f15e0d-527e-48d4-9638-acf7273aa57c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.139734 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f15e0d-527e-48d4-9638-acf7273aa57c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.149898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f15e0d-527e-48d4-9638-acf7273aa57c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:18:56.154696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-00f15e0d-527e-48d4-9638-acf7273aa57c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:18:56.154696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:18:56.154696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:18:56.154696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:18:56.154696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:18:56.154696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:18:56.154696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:18:56.154696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:18:56.154696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:18:56.154696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:18:56.154696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:18:56.154696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:18:56.154696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:18:56.154696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:18:56.154696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:18:56.154696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:18:56.155554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:18:56.155554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:18:56.155554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:18:56.155554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:18:56.155554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:18:56.155554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:18:56.155554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:18:56.155554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:18:56.155554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:18:56.155554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:18:56.155554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:18:56.155554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:18:56.155554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:18:56.155554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:18:56.155554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:18:56.155554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:18:56.156536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:18:56.156536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:18:56.156536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:18:56.156536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:18:56.156536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:18:56.156536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:18:56.156536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:18:56.156536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:18:56.156536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:18:56.156536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:18:56.156536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:18:56.156536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:18:56.156536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 38/80] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:18:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:18:56.157933 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f15e0d-527e-48d4-9638-acf7273aa57c None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.178032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:56.213875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc6fe4-3b96-416b-a82a-0e0a758353ee None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:56.246969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a11a7e-9ea8-4f9c-980b-70075294b7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:56.247574 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a11a7e-9ea8-4f9c-980b-70075294b7ab None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.247890 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a11a7e-9ea8-4f9c-980b-70075294b7ab None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.248184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a11a7e-9ea8-4f9c-980b-70075294b7ab None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.248845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a11a7e-9ea8-4f9c-980b-70075294b7ab None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:56.249158 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a11a7e-9ea8-4f9c-980b-70075294b7ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:56.250564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a11a7e-9ea8-4f9c-980b-70075294b7ab None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:56.256674 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f15e0d-527e-48d4-9638-acf7273aa57c None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:56.258440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 43/81] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:18:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:56.259229 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a11a7e-9ea8-4f9c-980b-70075294b7ab None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.294938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc5035-d20e-46f9-8b81-388fa6bea77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:56.295576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc5035-d20e-46f9-8b81-388fa6bea77a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.295889 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc5035-d20e-46f9-8b81-388fa6bea77a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.296171 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc5035-d20e-46f9-8b81-388fa6bea77a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.297274 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc5035-d20e-46f9-8b81-388fa6bea77a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:56.298737 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc5035-d20e-46f9-8b81-388fa6bea77a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:56.300346 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 39/82] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:18:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:56.301116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc5035-d20e-46f9-8b81-388fa6bea77a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.323915 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:56.360062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a11a7e-9ea8-4f9c-980b-70075294b7ab None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:56.386056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:56.394658 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9afb1-b271-4faf-8258-873c9b34ce84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:56.395298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9afb1-b271-4faf-8258-873c9b34ce84 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.395587 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9afb1-b271-4faf-8258-873c9b34ce84 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.395898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9afb1-b271-4faf-8258-873c9b34ce84 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.397039 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9afb1-b271-4faf-8258-873c9b34ce84 None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:56.398491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9afb1-b271-4faf-8258-873c9b34ce84 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:56.400173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 44/83] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:18:56 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:56.400943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9afb1-b271-4faf-8258-873c9b34ce84 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.401228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc5035-d20e-46f9-8b81-388fa6bea77a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:56.438173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbd83f-74a1-495c-b50a-041af713cafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:56.438707 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbd83f-74a1-495c-b50a-041af713cafb None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.438989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbd83f-74a1-495c-b50a-041af713cafb None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.439262 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbd83f-74a1-495c-b50a-041af713cafb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.502925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9afb1-b271-4faf-8258-873c9b34ce84 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:56.511443 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbd83f-74a1-495c-b50a-041af713cafb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:56.513032 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbd83f-74a1-495c-b50a-041af713cafb None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:56.533413 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:56.537040 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa06a4c-884a-49aa-809a-d16684c20e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:56.537677 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa06a4c-884a-49aa-809a-d16684c20e75 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.537964 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa06a4c-884a-49aa-809a-d16684c20e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.538245 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa06a4c-884a-49aa-809a-d16684c20e75 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.538911 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa06a4c-884a-49aa-809a-d16684c20e75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:56.539224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa06a4c-884a-49aa-809a-d16684c20e75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:56.540631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa06a4c-884a-49aa-809a-d16684c20e75 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:56.549320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 45/84] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:18:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:56.550191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa06a4c-884a-49aa-809a-d16684c20e75 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.609552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:56.618117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 40/85] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:18:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:18:56.618963 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbd83f-74a1-495c-b50a-041af713cafb None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.653732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa06a4c-884a-49aa-809a-d16684c20e75 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:56.686603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9b67e-cad0-43d5-9b70-b0063bf8f8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:56.687235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9b67e-cad0-43d5-9b70-b0063bf8f8eb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.687522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9b67e-cad0-43d5-9b70-b0063bf8f8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.687835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9b67e-cad0-43d5-9b70-b0063bf8f8eb None admin] PATH_INFO: `/v3/users/6c5badd8f28a4d749446523e9d5a6ed3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.689019 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9b67e-cad0-43d5-9b70-b0063bf8f8eb None admin] RBAC: Authorizing `identity:get_user(user_id=6c5badd8f28a4d749446523e9d5a6ed3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:56.690543 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9b67e-cad0-43d5-9b70-b0063bf8f8eb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:56.692201 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 46/86] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:18:56 2022] GET /identity/v3/users/6c5badd8f28a4d749446523e9d5a6ed3 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:56.692963 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9b67e-cad0-43d5-9b70-b0063bf8f8eb None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.727518 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbd83f-74a1-495c-b50a-041af713cafb None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:56.747720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:56.762294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3b41-44d2-4293-93d3-e72b7c226db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:56.762906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3b41-44d2-4293-93d3-e72b7c226db9 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.763188 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3b41-44d2-4293-93d3-e72b7c226db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.763472 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c3b41-44d2-4293-93d3-e72b7c226db9 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.764148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c3b41-44d2-4293-93d3-e72b7c226db9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:56.765623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c3b41-44d2-4293-93d3-e72b7c226db9 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:56.785663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 41/87] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:18:56 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:18:56.786483 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3b41-44d2-4293-93d3-e72b7c226db9 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.796744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9b67e-cad0-43d5-9b70-b0063bf8f8eb None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:56.828088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:56.846869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20a688-812f-48ec-8d38-2573af86b85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:56.847736 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20a688-812f-48ec-8d38-2573af86b85a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.848093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20a688-812f-48ec-8d38-2573af86b85a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.848443 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20a688-812f-48ec-8d38-2573af86b85a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.849266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20a688-812f-48ec-8d38-2573af86b85a None admin] RBAC: Adding query filter params (role.id=156b5bb1275a4dbeb61dbdade2c20d21, scope.project.id=c480d76c537348a7a86ae331f79d5e39, user.id=6c5badd8f28a4d749446523e9d5a6ed3) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:56.849651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20a688-812f-48ec-8d38-2573af86b85a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:56.851269 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad20a688-812f-48ec-8d38-2573af86b85a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:56.857730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 47/88] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:18:56 2022] GET /identity/v3/role_assignments?user.id=6c5badd8f28a4d749446523e9d5a6ed3&scope.project.id=c480d76c537348a7a86ae331f79d5e39&role.id=156b5bb1275a4dbeb61dbdade2c20d21 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:56.864920 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20a688-812f-48ec-8d38-2573af86b85a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.911353 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c3b41-44d2-4293-93d3-e72b7c226db9 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:56.958651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e84d81-8b62-473f-93d9-2c0b24704597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:56.959339 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e84d81-8b62-473f-93d9-2c0b24704597 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:56.959733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e84d81-8b62-473f-93d9-2c0b24704597 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:56.960017 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e84d81-8b62-473f-93d9-2c0b24704597 None admin] PATH_INFO: `/v3/users/6c5badd8f28a4d749446523e9d5a6ed3` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:56.961215 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e84d81-8b62-473f-93d9-2c0b24704597 None admin] RBAC: Authorizing `identity:get_user(user_id=6c5badd8f28a4d749446523e9d5a6ed3)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:56.962696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e84d81-8b62-473f-93d9-2c0b24704597 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:56.965088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 42/89] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:18:56 2022] GET /identity/v3/users/6c5badd8f28a4d749446523e9d5a6ed3 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:56.965920 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e84d81-8b62-473f-93d9-2c0b24704597 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:56.994870 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad20a688-812f-48ec-8d38-2573af86b85a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:57.026692 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:57.045495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358db115-3f97-475f-a835-6bdd2c64d7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:57.046139 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358db115-3f97-475f-a835-6bdd2c64d7ee None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:57.046418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358db115-3f97-475f-a835-6bdd2c64d7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:57.046693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358db115-3f97-475f-a835-6bdd2c64d7ee None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:57.048057 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358db115-3f97-475f-a835-6bdd2c64d7ee None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:57.049531 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358db115-3f97-475f-a835-6bdd2c64d7ee None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:57.051373 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 48/90] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:18:56 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:57.053007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358db115-3f97-475f-a835-6bdd2c64d7ee None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:57.105361 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e84d81-8b62-473f-93d9-2c0b24704597 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:57.133506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:57.156298 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29751a5-c473-43b4-ab11-0689e4b13545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:57.157077 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29751a5-c473-43b4-ab11-0689e4b13545 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:57.157455 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29751a5-c473-43b4-ab11-0689e4b13545 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:57.157915 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29751a5-c473-43b4-ab11-0689e4b13545 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:57.193434 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358db115-3f97-475f-a835-6bdd2c64d7ee None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:57.232639 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c023ec-e195-454f-a0bb-36e3c3027127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:57.233319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c023ec-e195-454f-a0bb-36e3c3027127 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:57.233711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c023ec-e195-454f-a0bb-36e3c3027127 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:57.234085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c023ec-e195-454f-a0bb-36e3c3027127 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:57.250441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29751a5-c473-43b4-ab11-0689e4b13545 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:57.252552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29751a5-c473-43b4-ab11-0689e4b13545 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:57.312672 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:57.320371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c023ec-e195-454f-a0bb-36e3c3027127 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:57.322162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c023ec-e195-454f-a0bb-36e3c3027127 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:57.382983 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 43/91] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:18:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:18:57.384092 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29751a5-c473-43b4-ab11-0689e4b13545 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:57.393325 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:57.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 49/92] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:18:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:18:57.455154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c023ec-e195-454f-a0bb-36e3c3027127 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:57.564046 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29751a5-c473-43b4-ab11-0689e4b13545 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:57.581826 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c023ec-e195-454f-a0bb-36e3c3027127 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:57.617451 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f05f2c-2c22-4c32-bab9-fce21f624092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:57.618094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f05f2c-2c22-4c32-bab9-fce21f624092 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:57.618380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f05f2c-2c22-4c32-bab9-fce21f624092 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:57.618659 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f05f2c-2c22-4c32-bab9-fce21f624092 None admin] PATH_INFO: `/v3/projects/5f4711b51fa54184981937491c080b51` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:57.619838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f05f2c-2c22-4c32-bab9-fce21f624092 None admin] RBAC: Authorizing `identity:get_project(project_id=5f4711b51fa54184981937491c080b51)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:57.621906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f05f2c-2c22-4c32-bab9-fce21f624092 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:57.623589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 50/93] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:18:57 2022] GET /identity/v3/projects/5f4711b51fa54184981937491c080b51 => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:57.624380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f05f2c-2c22-4c32-bab9-fce21f624092 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:57.635543 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92c53f-1b7a-4176-9155-74fb57afaf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:57.636205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92c53f-1b7a-4176-9155-74fb57afaf48 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:57.637599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92c53f-1b7a-4176-9155-74fb57afaf48 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:57.637972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92c53f-1b7a-4176-9155-74fb57afaf48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:57.638733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92c53f-1b7a-4176-9155-74fb57afaf48 None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=c480d76c537348a7a86ae331f79d5e39, user.id=ec5a3f02319e4ee5bdd1b0a6b24084c2) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:57.639049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92c53f-1b7a-4176-9155-74fb57afaf48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:57.640737 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92c53f-1b7a-4176-9155-74fb57afaf48 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:57.652802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:57.656991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 44/94] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:18:57 2022] GET /identity/v3/role_assignments?user.id=ec5a3f02319e4ee5bdd1b0a6b24084c2&scope.project.id=c480d76c537348a7a86ae331f79d5e39&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:57.659731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92c53f-1b7a-4176-9155-74fb57afaf48 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:57.679671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:57.736175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f05f2c-2c22-4c32-bab9-fce21f624092 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:57.781635 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ba261-0131-4a7c-8f23-0bb50d1b2d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:57.782809 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ba261-0131-4a7c-8f23-0bb50d1b2d71 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:57.783997 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ba261-0131-4a7c-8f23-0bb50d1b2d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:57.784667 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ba261-0131-4a7c-8f23-0bb50d1b2d71 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:57.788695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ba261-0131-4a7c-8f23-0bb50d1b2d71 None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:57.790475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ba261-0131-4a7c-8f23-0bb50d1b2d71 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:57.795276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 51/95] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:18:57 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:57.804835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ba261-0131-4a7c-8f23-0bb50d1b2d71 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:18:57.866182 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92c53f-1b7a-4176-9155-74fb57afaf48 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:57.935487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f63d9-c3d2-42d5-ac19-dd9c486890f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:57.936477 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f63d9-c3d2-42d5-ac19-dd9c486890f0 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:57.936956 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f63d9-c3d2-42d5-ac19-dd9c486890f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:57.937447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f63d9-c3d2-42d5-ac19-dd9c486890f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:57.938374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f63d9-c3d2-42d5-ac19-dd9c486890f0 None admin] RBAC: Adding query filter params (role.id=156b5bb1275a4dbeb61dbdade2c20d21, scope.project.id=5f4711b51fa54184981937491c080b51, user.id=6c5badd8f28a4d749446523e9d5a6ed3) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:57.938878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f63d9-c3d2-42d5-ac19-dd9c486890f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:57.940826 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f63d9-c3d2-42d5-ac19-dd9c486890f0 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:57.947410 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 45/96] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:18:57 2022] GET /identity/v3/role_assignments?user.id=6c5badd8f28a4d749446523e9d5a6ed3&scope.project.id=5f4711b51fa54184981937491c080b51&role.id=156b5bb1275a4dbeb61dbdade2c20d21 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:18:58.025005 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:58.037800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ba261-0131-4a7c-8f23-0bb50d1b2d71 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:18:58.085016 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531db43-ddea-471e-b460-0f265acfea9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:18:58.085642 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531db43-ddea-471e-b460-0f265acfea9c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:58.085926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531db43-ddea-471e-b460-0f265acfea9c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:58.086200 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531db43-ddea-471e-b460-0f265acfea9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:58.086947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531db43-ddea-471e-b460-0f265acfea9c None admin] RBAC: Adding query filter params (role.id=4e002a22056744acb8c40a3236cab404, scope.project.id=c480d76c537348a7a86ae331f79d5e39, user.id=6c5badd8f28a4d749446523e9d5a6ed3) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:18:58.087259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531db43-ddea-471e-b460-0f265acfea9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:18:58.088748 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531db43-ddea-471e-b460-0f265acfea9c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:18:58.100077 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 52/97] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:18:57 2022] GET /identity/v3/role_assignments?user.id=6c5badd8f28a4d749446523e9d5a6ed3&scope.project.id=c480d76c537348a7a86ae331f79d5e39&role.id=4e002a22056744acb8c40a3236cab404 => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:18:59.467788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d232f-f6a7-4207-bbad-3c23a4d6c72b None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:59.468302 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d232f-f6a7-4207-bbad-3c23a4d6c72b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:59.468375 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d232f-f6a7-4207-bbad-3c23a4d6c72b None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:59.469862 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 46/98] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:59.475146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f7e0f-a114-4f3b-8373-bd1b34a23e3b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:59.475569 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f7e0f-a114-4f3b-8373-bd1b34a23e3b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:59.475757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f7e0f-a114-4f3b-8373-bd1b34a23e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:59.495322 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-228f7e0f-a114-4f3b-8373-bd1b34a23e3b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:59.497762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-228f7e0f-a114-4f3b-8373-bd1b34a23e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:59.528576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d1d54-e6b0-4d02-99c4-416a01a7d3e9 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:59.528908 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d1d54-e6b0-4d02-99c4-416a01a7d3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:59.529139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d1d54-e6b0-4d02-99c4-416a01a7d3e9 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:59.530538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 47/99] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:18:59.535964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a2894-6764-44ec-9da4-8b0a938d1446 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:59.536257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a2894-6764-44ec-9da4-8b0a938d1446 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:59.536539 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a2894-6764-44ec-9da4-8b0a938d1446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:59.560329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-774a2894-6764-44ec-9da4-8b0a938d1446 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:59.561829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-774a2894-6764-44ec-9da4-8b0a938d1446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:59.588708 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 53/100] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:18:59.675583 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 48/101] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:18:59.727733 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f3458-80c0-4581-b8d5-abe21d751d03 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:59.728004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f3458-80c0-4581-b8d5-abe21d751d03 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:59.728206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f3458-80c0-4581-b8d5-abe21d751d03 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:59.729695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 54/102] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:18:59.744373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb3f16-de06-48b1-98a9-439a4c6a7869 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:18:59.744673 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb3f16-de06-48b1-98a9-439a4c6a7869 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:18:59.744950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb3f16-de06-48b1-98a9-439a4c6a7869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:18:59.754961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:18:59.772599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-badb3f16-de06-48b1-98a9-439a4c6a7869 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:18:59.773813 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-badb3f16-de06-48b1-98a9-439a4c6a7869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:18:59.871566 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 49/103] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:18:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:00.088175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f3458-80c0-4581-b8d5-abe21d751d03 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:00.177340 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb3f16-de06-48b1-98a9-439a4c6a7869 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:00.195800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:00.216009 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f3458-80c0-4581-b8d5-abe21d751d03 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:00.293926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3f357-d0c3-4b06-b37e-aea50e55feb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:00.294583 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3f357-d0c3-4b06-b37e-aea50e55feb2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:00.294872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3f357-d0c3-4b06-b37e-aea50e55feb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:00.295152 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad3f357-d0c3-4b06-b37e-aea50e55feb2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:00.314640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad3f357-d0c3-4b06-b37e-aea50e55feb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:00.320127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-5ad3f357-d0c3-4b06-b37e-aea50e55feb2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:00.320127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:00.320127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:00.320127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:00.320127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:00.320127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:00.320127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:00.320127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:00.320127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:00.320127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:00.320127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:00.320127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:00.320127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:00.320127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:00.320127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:00.320127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:00.321264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:00.321264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:00.321264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:00.321264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:00.321264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:00.321264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:00.321264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:00.321264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:00.321264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:00.321264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:00.321264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:00.321264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:00.321264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:00.321264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:00.321264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:00.321264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:00.322240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:00.322240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:00.322240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:00.322240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:00.322240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:00.322240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:00.322240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:00.322240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:00.322240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:00.322240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:00.322240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:00.322240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:00.324601 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 55/104] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:00.325581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3f357-d0c3-4b06-b37e-aea50e55feb2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:00.389001 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb3f16-de06-48b1-98a9-439a4c6a7869 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:00.443996 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:00.453003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc8ebd-e0d3-4afa-ad5d-661c263a4227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:00.453653 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc8ebd-e0d3-4afa-ad5d-661c263a4227 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:00.453940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc8ebd-e0d3-4afa-ad5d-661c263a4227 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:00.454220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc8ebd-e0d3-4afa-ad5d-661c263a4227 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:00.471532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc8ebd-e0d3-4afa-ad5d-661c263a4227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:00.476402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-17bc8ebd-e0d3-4afa-ad5d-661c263a4227 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:00.476402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:00.476402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:00.476402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:00.476402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:00.476402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:00.476402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:00.476402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:00.476402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:00.476402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:00.476402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:00.476402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:00.476402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:00.476402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:00.476402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:00.476402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:00.477440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:00.477440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:00.477440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:00.477440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:00.477440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:00.477440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:00.477440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:00.477440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:00.477440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:00.477440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:00.477440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:00.477440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:00.477440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:00.477440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:00.477440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:00.477440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:00.478390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:00.478390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:00.478390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:00.478390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:00.478390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:00.478390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:00.478390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:00.478390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:00.478390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:00.478390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:00.478390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:00.478390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:00.482947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 50/105] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:00.485087 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc8ebd-e0d3-4afa-ad5d-661c263a4227 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:00.505983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad3f357-d0c3-4b06-b37e-aea50e55feb2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:00.568951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b1004-cfa0-4d3c-a2a4-c98e19e634bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:00.569920 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b1004-cfa0-4d3c-a2a4-c98e19e634bf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:00.570430 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b1004-cfa0-4d3c-a2a4-c98e19e634bf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:00.571063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b1004-cfa0-4d3c-a2a4-c98e19e634bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:00.573830 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b1004-cfa0-4d3c-a2a4-c98e19e634bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:00.576393 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b1004-cfa0-4d3c-a2a4-c98e19e634bf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:00.579982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 56/106] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:00.582472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b1004-cfa0-4d3c-a2a4-c98e19e634bf None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:00.628052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:00.676751 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc8ebd-e0d3-4afa-ad5d-661c263a4227 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:00.730869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a661fe5-8df0-4c01-9c48-4cf7031127a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:00.731818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a661fe5-8df0-4c01-9c48-4cf7031127a7 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:00.732273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a661fe5-8df0-4c01-9c48-4cf7031127a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:00.732895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a661fe5-8df0-4c01-9c48-4cf7031127a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:00.733914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a661fe5-8df0-4c01-9c48-4cf7031127a7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:00.734397 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a661fe5-8df0-4c01-9c48-4cf7031127a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:00.736607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a661fe5-8df0-4c01-9c48-4cf7031127a7 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:00.748592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 51/107] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:00.749925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a661fe5-8df0-4c01-9c48-4cf7031127a7 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:00.760529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b1004-cfa0-4d3c-a2a4-c98e19e634bf None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:00.807856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c75f72-7a47-4879-bdd5-35151d9ca86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:00.809295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c75f72-7a47-4879-bdd5-35151d9ca86c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:00.809728 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c75f72-7a47-4879-bdd5-35151d9ca86c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:00.810408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c75f72-7a47-4879-bdd5-35151d9ca86c None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:00.811744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c75f72-7a47-4879-bdd5-35151d9ca86c None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:00.812200 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c75f72-7a47-4879-bdd5-35151d9ca86c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:00.814270 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c75f72-7a47-4879-bdd5-35151d9ca86c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:00.822712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:00.826955 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 57/108] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:00.829083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bb3ac-c10c-4234-898a-d07572cb6d85 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:00.829376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bb3ac-c10c-4234-898a-d07572cb6d85 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:00.829656 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80bb3ac-c10c-4234-898a-d07572cb6d85 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:00.835043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 58/109] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:00.835043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80bb3ac-c10c-4234-898a-d07572cb6d85 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:00.890700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a661fe5-8df0-4c01-9c48-4cf7031127a7 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:00.926747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7106d-4319-4ec8-a4f9-4144ed7866bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:00.927683 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7106d-4319-4ec8-a4f9-4144ed7866bd None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:00.927912 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7106d-4319-4ec8-a4f9-4144ed7866bd None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:00.928197 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7106d-4319-4ec8-a4f9-4144ed7866bd None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:00.928846 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf7106d-4319-4ec8-a4f9-4144ed7866bd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:00.930307 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf7106d-4319-4ec8-a4f9-4144ed7866bd None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:00.938680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:00.959521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 52/110] 10.222.0.58 () {60 vars in 1141 bytes} [Wed Aug 24 12:19:00 2022] POST /identity/v3/projects => generated 302 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:00.960496 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7106d-4319-4ec8-a4f9-4144ed7866bd None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:00.975589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80bb3ac-c10c-4234-898a-d07572cb6d85 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:01.029073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba27726-5e50-47e6-b10f-9c20f947dd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:01.030912 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba27726-5e50-47e6-b10f-9c20f947dd65 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:01.031567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba27726-5e50-47e6-b10f-9c20f947dd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:01.031883 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba27726-5e50-47e6-b10f-9c20f947dd65 None admin] PATH_INFO: `/v3/users/fc82f935e3a047f58c93d87ad0d07ecc` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:01.056529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba27726-5e50-47e6-b10f-9c20f947dd65 None admin] RBAC: Authorizing `identity:get_user(user_id=fc82f935e3a047f58c93d87ad0d07ecc)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:01.058962 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba27726-5e50-47e6-b10f-9c20f947dd65 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:01.061641 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 59/111] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:00 2022] GET /identity/v3/users/fc82f935e3a047f58c93d87ad0d07ecc => generated 289 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:01.062945 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb9888-5550-48f7-ba46-47f648438217 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:01.063241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb9888-5550-48f7-ba46-47f648438217 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:01.063525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb9888-5550-48f7-ba46-47f648438217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:01.081545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7106d-4319-4ec8-a4f9-4144ed7866bd None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:01.086159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-57fb9888-5550-48f7-ba46-47f648438217 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:01.087500 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-57fb9888-5550-48f7-ba46-47f648438217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:01.091367 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:01.127578 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73906171-22e5-473a-8cf9-809999a720f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:01.128921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73906171-22e5-473a-8cf9-809999a720f9 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:01.129220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73906171-22e5-473a-8cf9-809999a720f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:01.129497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73906171-22e5-473a-8cf9-809999a720f9 None admin] PATH_INFO: `/v3/users/6c5badd8f28a4d749446523e9d5a6ed3` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:01.130672 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73906171-22e5-473a-8cf9-809999a720f9 None admin] RBAC: Authorizing `identity:get_user(user_id=6c5badd8f28a4d749446523e9d5a6ed3)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:01.132173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73906171-22e5-473a-8cf9-809999a720f9 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:01.133919 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 53/112] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:00 2022] GET /identity/v3/users/6c5badd8f28a4d749446523e9d5a6ed3 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:01.134941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc81a03-3078-4657-9b6a-abafd4015e85 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:01.135248 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc81a03-3078-4657-9b6a-abafd4015e85 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:01.135499 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc81a03-3078-4657-9b6a-abafd4015e85 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:01.137485 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 54/113] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:01.138088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc81a03-3078-4657-9b6a-abafd4015e85 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:01.176236 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 60/114] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:01.178932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899188df-e4a2-465b-b431-d6e2e656779e None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:01.179173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899188df-e4a2-465b-b431-d6e2e656779e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:01.179453 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899188df-e4a2-465b-b431-d6e2e656779e None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:01.182731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 61/115] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:01.184297 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899188df-e4a2-465b-b431-d6e2e656779e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:01.255963 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:01.281496 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc81a03-3078-4657-9b6a-abafd4015e85 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:01.297420 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899188df-e4a2-465b-b431-d6e2e656779e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:01.337598 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a6716-3156-45d7-9f18-98d7097e9af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:01.338228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a6716-3156-45d7-9f18-98d7097e9af1 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:01.338513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a6716-3156-45d7-9f18-98d7097e9af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:01.338789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a6716-3156-45d7-9f18-98d7097e9af1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:01.347201 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea7e3b-a177-47f2-bead-eb2d5bfbdc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:01.347876 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea7e3b-a177-47f2-bead-eb2d5bfbdc95 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:01.348159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea7e3b-a177-47f2-bead-eb2d5bfbdc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:01.348433 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea7e3b-a177-47f2-bead-eb2d5bfbdc95 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:01.349574 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea7e3b-a177-47f2-bead-eb2d5bfbdc95 None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:01.351049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea7e3b-a177-47f2-bead-eb2d5bfbdc95 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:01.352737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 62/116] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:01 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:01.353857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ca371-e69e-4f08-b168-d4097b61793a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:01.354146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ca371-e69e-4f08-b168-d4097b61793a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:01.354426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ca371-e69e-4f08-b168-d4097b61793a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:01.373051 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2b9ca371-e69e-4f08-b168-d4097b61793a None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:01.374248 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2b9ca371-e69e-4f08-b168-d4097b61793a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:01.400528 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:01.437404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a6716-3156-45d7-9f18-98d7097e9af1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:01.439932 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a6716-3156-45d7-9f18-98d7097e9af1 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:01.455139 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 63/117] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:01.456797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b7649-eafc-44ac-aec3-7be681eb290e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:01.457330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b7649-eafc-44ac-aec3-7be681eb290e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:01.457782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b7649-eafc-44ac-aec3-7be681eb290e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:01.480947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-193b7649-eafc-44ac-aec3-7be681eb290e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:01.482592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-193b7649-eafc-44ac-aec3-7be681eb290e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:01.558080 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:01.571436 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 55/118] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:01.572759 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 64/119] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:01.582907 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a6716-3156-45d7-9f18-98d7097e9af1 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:01.583048 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b7649-eafc-44ac-aec3-7be681eb290e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:01.705163 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:01.744588 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a6716-3156-45d7-9f18-98d7097e9af1 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:01.765202 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b7649-eafc-44ac-aec3-7be681eb290e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:01.804703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab8aad-e86c-43b7-a0d2-6aff9a216e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:01.805375 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab8aad-e86c-43b7-a0d2-6aff9a216e2e None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:01.805662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab8aad-e86c-43b7-a0d2-6aff9a216e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:01.805943 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab8aad-e86c-43b7-a0d2-6aff9a216e2e None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39/users/6c5badd8f28a4d749446523e9d5a6ed3/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:01.807956 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab8aad-e86c-43b7-a0d2-6aff9a216e2e None admin] RBAC: Authorizing `identity:create_grant(project_id=c480d76c537348a7a86ae331f79d5e39, user_id=6c5badd8f28a4d749446523e9d5a6ed3, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:01.810270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab8aad-e86c-43b7-a0d2-6aff9a216e2e None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:01.817433 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09210c34-1ebb-4e9d-835c-c939382df7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:01.818221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09210c34-1ebb-4e9d-835c-c939382df7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:01.818636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09210c34-1ebb-4e9d-835c-c939382df7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:01.819004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09210c34-1ebb-4e9d-835c-c939382df7f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:01.826707 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 56/120] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:01 2022] PUT /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39/users/6c5badd8f28a4d749446523e9d5a6ed3/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:01.827842 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab8aad-e86c-43b7-a0d2-6aff9a216e2e None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:01.832274 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09210c34-1ebb-4e9d-835c-c939382df7f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:01.837949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-09210c34-1ebb-4e9d-835c-c939382df7f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:01.837949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:01.837949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:01.837949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:01.837949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:01.837949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:01.837949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:01.837949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:01.837949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:01.837949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:01.837949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:01.837949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:01.837949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:01.837949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:01.837949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:01.837949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:01.838906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:01.838906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:01.838906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:01.838906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:01.838906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:01.838906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:01.838906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:01.838906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:01.838906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:01.838906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:01.838906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:01.838906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:01.838906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:01.838906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:01.838906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:01.838906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:01.840218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:01.840218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:01.840218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:01.840218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:01.840218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:01.840218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:01.840218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:01.840218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:01.840218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:01.840218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:01.840218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:01.840218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:01.840218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 65/121] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:01.840218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09210c34-1ebb-4e9d-835c-c939382df7f6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:01.937251 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:01.962813 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab8aad-e86c-43b7-a0d2-6aff9a216e2e None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:01.992855 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09210c34-1ebb-4e9d-835c-c939382df7f6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:02.007944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cdc21-d4ff-426a-bc3e-40d7f525ab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:02.008692 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cdc21-d4ff-426a-bc3e-40d7f525ab93 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:02.009052 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cdc21-d4ff-426a-bc3e-40d7f525ab93 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:02.009401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cdc21-d4ff-426a-bc3e-40d7f525ab93 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:02.010890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cdc21-d4ff-426a-bc3e-40d7f525ab93 None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:02.013250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cdc21-d4ff-426a-bc3e-40d7f525ab93 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:02.015179 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 57/122] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:01 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:02.017664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cdc21-d4ff-426a-bc3e-40d7f525ab93 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:02.020085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:02.035403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae3433-315b-4309-8e28-09f89e122b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:02.036085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae3433-315b-4309-8e28-09f89e122b43 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:02.036374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae3433-315b-4309-8e28-09f89e122b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:02.036913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae3433-315b-4309-8e28-09f89e122b43 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:02.051865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ae3433-315b-4309-8e28-09f89e122b43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:02.057492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-c3ae3433-315b-4309-8e28-09f89e122b43 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:02.057492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:02.057492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:02.057492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:02.057492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:02.057492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:02.057492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:02.057492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:02.057492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:02.057492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:02.057492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:02.057492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:02.057492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:02.057492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:02.057492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:02.057492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:02.059037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:02.059037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:02.059037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:02.059037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:02.059037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:02.059037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:02.059037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:02.059037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:02.059037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:02.059037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:02.059037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:02.059037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:02.059037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:02.059037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:02.059037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:02.059037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:02.060352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:02.060352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:02.060352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:02.060352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:02.060352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:02.060352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:02.060352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:02.060352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:02.060352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:02.060352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:02.060352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:02.060352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:02.060352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 66/123] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:02.060352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae3433-315b-4309-8e28-09f89e122b43 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:02.132001 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cdc21-d4ff-426a-bc3e-40d7f525ab93 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:02.165435 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae3433-315b-4309-8e28-09f89e122b43 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:02.171333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f8249-ef9e-403f-8f1d-c9facad748a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:02.172174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f8249-ef9e-403f-8f1d-c9facad748a3 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:02.172644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f8249-ef9e-403f-8f1d-c9facad748a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:02.173082 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f8249-ef9e-403f-8f1d-c9facad748a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:02.173998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f8249-ef9e-403f-8f1d-c9facad748a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:02.174442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f8249-ef9e-403f-8f1d-c9facad748a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:02.176438 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f8249-ef9e-403f-8f1d-c9facad748a3 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:02.181207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:02.188531 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 58/124] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:02.190528 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f8249-ef9e-403f-8f1d-c9facad748a3 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:02.204806 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0befa72-004e-4ca4-b983-766035099d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:02.205540 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0befa72-004e-4ca4-b983-766035099d0b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:02.205826 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0befa72-004e-4ca4-b983-766035099d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:02.206106 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0befa72-004e-4ca4-b983-766035099d0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:02.217975 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0befa72-004e-4ca4-b983-766035099d0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:02.222824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e0befa72-004e-4ca4-b983-766035099d0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:02.222824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:02.222824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:02.222824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:02.222824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:02.222824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:02.222824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:02.222824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:02.222824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:02.222824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:02.222824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:02.222824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:02.222824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:02.222824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:02.222824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:02.222824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:02.223922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:02.223922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:02.223922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:02.223922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:02.223922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:02.223922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:02.223922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:02.223922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:02.223922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:02.223922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:02.223922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:02.223922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:02.223922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:02.223922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:02.223922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:02.223922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:02.224838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:02.224838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:02.224838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:02.224838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:02.224838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:02.224838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:02.224838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:02.224838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:02.224838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:02.224838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:02.224838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:02.224838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:02.224838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 67/125] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:02.224838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0befa72-004e-4ca4-b983-766035099d0b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:02.263197 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:02.307213 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f8249-ef9e-403f-8f1d-c9facad748a3 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:02.330091 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0befa72-004e-4ca4-b983-766035099d0b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:02.346572 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72a82a-5c31-42e7-baab-857c264a73d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:02.347449 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72a82a-5c31-42e7-baab-857c264a73d2 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:02.348012 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72a82a-5c31-42e7-baab-857c264a73d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:02.348498 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72a82a-5c31-42e7-baab-857c264a73d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:02.349472 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72a82a-5c31-42e7-baab-857c264a73d2 None admin] RBAC: Adding query filter params (role.id=0f1253d845894622b8091c95dfd079d3, scope.project.id=c480d76c537348a7a86ae331f79d5e39, user.id=fc82f935e3a047f58c93d87ad0d07ecc) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:02.349958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72a82a-5c31-42e7-baab-857c264a73d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:02.351704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c72a82a-5c31-42e7-baab-857c264a73d2 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:02.358821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 59/126] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:02 2022] GET /identity/v3/role_assignments?user.id=fc82f935e3a047f58c93d87ad0d07ecc&scope.project.id=c480d76c537348a7a86ae331f79d5e39&role.id=0f1253d845894622b8091c95dfd079d3 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:02.359865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72a82a-5c31-42e7-baab-857c264a73d2 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:02.385371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1244c06-f358-423c-b634-d0e1b679f284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:02.386318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1244c06-f358-423c-b634-d0e1b679f284 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:02.386867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1244c06-f358-423c-b634-d0e1b679f284 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:02.387477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1244c06-f358-423c-b634-d0e1b679f284 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:02.388567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1244c06-f358-423c-b634-d0e1b679f284 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:02.389137 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1244c06-f358-423c-b634-d0e1b679f284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:02.393678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1244c06-f358-423c-b634-d0e1b679f284 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:02.416288 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 68/127] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:02.417486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1244c06-f358-423c-b634-d0e1b679f284 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:02.508203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:02.557503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72a82a-5c31-42e7-baab-857c264a73d2 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:02.610533 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:02.626806 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12b76e-9e44-4004-ad15-0efa02f88b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:02.628771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12b76e-9e44-4004-ad15-0efa02f88b51 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:02.629193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12b76e-9e44-4004-ad15-0efa02f88b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:02.630347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df12b76e-9e44-4004-ad15-0efa02f88b51 None admin] PATH_INFO: `/v3/users/ec5a3f02319e4ee5bdd1b0a6b24084c2` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:02.632727 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12b76e-9e44-4004-ad15-0efa02f88b51 None admin] RBAC: Authorizing `identity:get_user(user_id=ec5a3f02319e4ee5bdd1b0a6b24084c2)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:02.636758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df12b76e-9e44-4004-ad15-0efa02f88b51 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:02.642060 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1244c06-f358-423c-b634-d0e1b679f284 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:02.642907 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 60/128] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:02 2022] GET /identity/v3/users/ec5a3f02319e4ee5bdd1b0a6b24084c2 => generated 247 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:02.643810 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12b76e-9e44-4004-ad15-0efa02f88b51 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:02.713691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acc46f-922a-4411-b0e7-f636b550d7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:02.714537 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acc46f-922a-4411-b0e7-f636b550d7c9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:02.715582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acc46f-922a-4411-b0e7-f636b550d7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:02.716003 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acc46f-922a-4411-b0e7-f636b550d7c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:02.716809 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acc46f-922a-4411-b0e7-f636b550d7c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:02.717279 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acc46f-922a-4411-b0e7-f636b550d7c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:02.718798 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acc46f-922a-4411-b0e7-f636b550d7c9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:02.729116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 69/129] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:02.730296 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acc46f-922a-4411-b0e7-f636b550d7c9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:02.853487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df12b76e-9e44-4004-ad15-0efa02f88b51 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:02.920579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:02.924660 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acc46f-922a-4411-b0e7-f636b550d7c9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:02.939684 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8857e87-f447-48f3-9eea-11a1b94d6f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:02.940621 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8857e87-f447-48f3-9eea-11a1b94d6f20 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:02.941135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8857e87-f447-48f3-9eea-11a1b94d6f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:02.941580 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8857e87-f447-48f3-9eea-11a1b94d6f20 None admin] PATH_INFO: `/v3/users/6c5badd8f28a4d749446523e9d5a6ed3` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:02.948302 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8857e87-f447-48f3-9eea-11a1b94d6f20 None admin] RBAC: Authorizing `identity:get_user(user_id=6c5badd8f28a4d749446523e9d5a6ed3)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:02.950588 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8857e87-f447-48f3-9eea-11a1b94d6f20 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:02.952442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 61/130] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:02 2022] GET /identity/v3/users/6c5badd8f28a4d749446523e9d5a6ed3 => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:02.953410 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8857e87-f447-48f3-9eea-11a1b94d6f20 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:02.963847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b865b6-2845-48ed-abc1-a1ed496982e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:02.964493 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b865b6-2845-48ed-abc1-a1ed496982e3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:02.965445 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b865b6-2845-48ed-abc1-a1ed496982e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:02.965650 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b865b6-2845-48ed-abc1-a1ed496982e3 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:02.966800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b865b6-2845-48ed-abc1-a1ed496982e3 None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:02.971645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b865b6-2845-48ed-abc1-a1ed496982e3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:02.971645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 70/131] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:02 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:02.971645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b865b6-2845-48ed-abc1-a1ed496982e3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:03.065527 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:03.101011 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b865b6-2845-48ed-abc1-a1ed496982e3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:03.111281 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8857e87-f447-48f3-9eea-11a1b94d6f20 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:03.157525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f78e6-cdb4-4297-9900-01096aca1df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:03.158643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f78e6-cdb4-4297-9900-01096aca1df4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.159233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f78e6-cdb4-4297-9900-01096aca1df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.159951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f78e6-cdb4-4297-9900-01096aca1df4 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.161963 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f78e6-cdb4-4297-9900-01096aca1df4 None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:03.164714 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f78e6-cdb4-4297-9900-01096aca1df4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:03.168155 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 71/132] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:02 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:03.169754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f78e6-cdb4-4297-9900-01096aca1df4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:03.175222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d693ebb-5dd4-4e0f-ac23-39121846a42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:03.177176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d693ebb-5dd4-4e0f-ac23-39121846a42e None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.177660 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d693ebb-5dd4-4e0f-ac23-39121846a42e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.178146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d693ebb-5dd4-4e0f-ac23-39121846a42e None admin] PATH_INFO: `/v3/users/6c5badd8f28a4d749446523e9d5a6ed3` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.180219 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d693ebb-5dd4-4e0f-ac23-39121846a42e None admin] RBAC: Authorizing `identity:get_user(user_id=6c5badd8f28a4d749446523e9d5a6ed3)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:03.182770 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d693ebb-5dd4-4e0f-ac23-39121846a42e None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:03.185894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 62/133] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:02 2022] GET /identity/v3/users/6c5badd8f28a4d749446523e9d5a6ed3 => generated 275 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:03.187809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be147e7-df54-499f-90c9-48c1b6a909b1 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.188375 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be147e7-df54-499f-90c9-48c1b6a909b1 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.188879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be147e7-df54-499f-90c9-48c1b6a909b1 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.191182 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 63/134] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:03.193225 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be147e7-df54-499f-90c9-48c1b6a909b1 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:03.237008 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:03.335306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f78e6-cdb4-4297-9900-01096aca1df4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:03.351611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be147e7-df54-499f-90c9-48c1b6a909b1 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:03.378053 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d5880-9385-4624-88bd-0969b25f9142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:03.379083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d5880-9385-4624-88bd-0969b25f9142 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.379373 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d5880-9385-4624-88bd-0969b25f9142 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.379965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d5880-9385-4624-88bd-0969b25f9142 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39/users/ec5a3f02319e4ee5bdd1b0a6b24084c2/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.382410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d5880-9385-4624-88bd-0969b25f9142 None admin] RBAC: Authorizing `identity:create_grant(project_id=c480d76c537348a7a86ae331f79d5e39, user_id=ec5a3f02319e4ee5bdd1b0a6b24084c2, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:03.384395 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d5880-9385-4624-88bd-0969b25f9142 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:03.389065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:03.396819 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d9206-77a3-4bc4-8783-6237f10a39ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:03.397790 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d9206-77a3-4bc4-8783-6237f10a39ab None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.398297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d9206-77a3-4bc4-8783-6237f10a39ab None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.398758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d9206-77a3-4bc4-8783-6237f10a39ab None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39/users/6c5badd8f28a4d749446523e9d5a6ed3/roles/4e002a22056744acb8c40a3236cab404` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.401546 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d9206-77a3-4bc4-8783-6237f10a39ab None admin] RBAC: Authorizing `identity:create_grant(project_id=c480d76c537348a7a86ae331f79d5e39, user_id=6c5badd8f28a4d749446523e9d5a6ed3, role_id=4e002a22056744acb8c40a3236cab404)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:03.404329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d9206-77a3-4bc4-8783-6237f10a39ab None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:03.407688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 72/135] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:03 2022] PUT /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39/users/ec5a3f02319e4ee5bdd1b0a6b24084c2/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:03.408768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d5880-9385-4624-88bd-0969b25f9142 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:03.436572 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 64/136] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:03 2022] PUT /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39/users/6c5badd8f28a4d749446523e9d5a6ed3/roles/4e002a22056744acb8c40a3236cab404 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:03.438361 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661d2d4-f0ad-46f7-8404-c99339c734f7 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.438880 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661d2d4-f0ad-46f7-8404-c99339c734f7 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.440545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661d2d4-f0ad-46f7-8404-c99339c734f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.479027 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-7661d2d4-f0ad-46f7-8404-c99339c734f7 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:03.483636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7661d2d4-f0ad-46f7-8404-c99339c734f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:03.599234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 65/137] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:03.624108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:03.630062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d5880-9385-4624-88bd-0969b25f9142 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:03.667123 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88edb2-2d37-41c2-bb55-8173265babdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:03.668703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88edb2-2d37-41c2-bb55-8173265babdd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.669001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88edb2-2d37-41c2-bb55-8173265babdd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.669277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88edb2-2d37-41c2-bb55-8173265babdd None admin] PATH_INFO: `/v3/projects/5f4711b51fa54184981937491c080b51` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.670595 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88edb2-2d37-41c2-bb55-8173265babdd None admin] RBAC: Authorizing `identity:get_project(project_id=5f4711b51fa54184981937491c080b51)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:03.672646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88edb2-2d37-41c2-bb55-8173265babdd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:03.676090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 73/138] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:03 2022] GET /identity/v3/projects/5f4711b51fa54184981937491c080b51 => generated 312 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:03.681319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661d2d4-f0ad-46f7-8404-c99339c734f7 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:03.798233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661d2d4-f0ad-46f7-8404-c99339c734f7 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:03.822273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:03.842232 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694d163-0027-417a-9e52-d7b71c70eb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:03.842911 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694d163-0027-417a-9e52-d7b71c70eb88 None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:03.843192 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694d163-0027-417a-9e52-d7b71c70eb88 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:03.843467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694d163-0027-417a-9e52-d7b71c70eb88 None admin] PATH_INFO: `/v3/projects/5f4711b51fa54184981937491c080b51/users/6c5badd8f28a4d749446523e9d5a6ed3/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:03.845459 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5694d163-0027-417a-9e52-d7b71c70eb88 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f4711b51fa54184981937491c080b51, user_id=6c5badd8f28a4d749446523e9d5a6ed3, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:03.846985 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5694d163-0027-417a-9e52-d7b71c70eb88 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:03.876922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 66/139] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:03 2022] PUT /identity/v3/projects/5f4711b51fa54184981937491c080b51/users/6c5badd8f28a4d749446523e9d5a6ed3/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:03.993692 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88edb2-2d37-41c2-bb55-8173265babdd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:04.113885 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88edb2-2d37-41c2-bb55-8173265babdd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:04.150825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0345dc-e6ba-4760-aeb7-eef5b53e4993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:04.151395 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0345dc-e6ba-4760-aeb7-eef5b53e4993 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.151701 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0345dc-e6ba-4760-aeb7-eef5b53e4993 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.151984 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0345dc-e6ba-4760-aeb7-eef5b53e4993 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.154141 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0345dc-e6ba-4760-aeb7-eef5b53e4993 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:04.155736 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0345dc-e6ba-4760-aeb7-eef5b53e4993 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:04.157372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 74/140] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:04.163226 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694d163-0027-417a-9e52-d7b71c70eb88 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:04.188720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ab6ce-25da-4e9d-be4a-e0221bd040e2 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.189020 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ab6ce-25da-4e9d-be4a-e0221bd040e2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.189276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ab6ce-25da-4e9d-be4a-e0221bd040e2 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.191005 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 75/141] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:04.196833 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851fd84-7b8e-45a4-85f8-3db4576faf9b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.197117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851fd84-7b8e-45a4-85f8-3db4576faf9b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.197394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851fd84-7b8e-45a4-85f8-3db4576faf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.223393 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f851fd84-7b8e-45a4-85f8-3db4576faf9b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:04.224637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f851fd84-7b8e-45a4-85f8-3db4576faf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:04.276002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:04.339233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 76/142] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:04.360066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694d163-0027-417a-9e52-d7b71c70eb88 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:04.407149 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81215edd-b741-49ad-bdae-31d4e5454ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:04.407712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81215edd-b741-49ad-bdae-31d4e5454ea3 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.407999 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81215edd-b741-49ad-bdae-31d4e5454ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.408276 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81215edd-b741-49ad-bdae-31d4e5454ea3 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.408943 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81215edd-b741-49ad-bdae-31d4e5454ea3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:04.410386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81215edd-b741-49ad-bdae-31d4e5454ea3 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:04.444942 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 67/143] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:04 2022] POST /identity/v3/users => generated 283 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:04.943549 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851fd84-7b8e-45a4-85f8-3db4576faf9b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:04.962940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832c19a-127a-49a2-885f-765ff7b02dbc None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.963444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832c19a-127a-49a2-885f-765ff7b02dbc None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.963532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832c19a-127a-49a2-885f-765ff7b02dbc None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.965505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 68/144] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:19:04.970825 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbaf32-4cae-470e-aba9-4db18ee76fb0 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:04.971116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbaf32-4cae-470e-aba9-4db18ee76fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:04.971401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbaf32-4cae-470e-aba9-4db18ee76fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:04.991584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-1cbbaf32-4cae-470e-aba9-4db18ee76fb0 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:04.999650 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1cbbaf32-4cae-470e-aba9-4db18ee76fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:05.024323 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:05.067288 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 69/145] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:05.096238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851fd84-7b8e-45a4-85f8-3db4576faf9b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:05.147908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d41b4-30f9-4ccd-83b7-56877882dedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:05.148475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d41b4-30f9-4ccd-83b7-56877882dedf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:05.148755 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d41b4-30f9-4ccd-83b7-56877882dedf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:05.149029 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d41b4-30f9-4ccd-83b7-56877882dedf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:05.173998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d41b4-30f9-4ccd-83b7-56877882dedf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:05.182603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-c69d41b4-30f9-4ccd-83b7-56877882dedf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:05.182603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:05.182603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:05.182603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:05.182603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:05.182603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:05.182603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:05.182603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:05.182603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:05.182603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:05.182603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:05.182603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:05.182603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:05.182603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:05.182603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:05.182603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:05.184365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:05.184365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:05.184365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:05.184365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:05.184365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:05.184365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:05.184365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:05.184365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:05.184365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:05.184365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:05.184365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:05.184365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:05.184365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:05.184365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:05.184365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:05.184365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:05.185134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:05.185134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:05.185134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:05.185134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:05.185134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:05.185134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:05.185134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:05.185134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:05.185134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:05.185134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:05.185134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:05.185134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:05.188000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 77/146] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:05.192864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbaf32-4cae-470e-aba9-4db18ee76fb0 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:05.322923 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbaf32-4cae-470e-aba9-4db18ee76fb0 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:05.360083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4d206-1cf2-407e-9960-96ef091aafb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:05.360940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4d206-1cf2-407e-9960-96ef091aafb0 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:05.361396 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4d206-1cf2-407e-9960-96ef091aafb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:05.361816 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4d206-1cf2-407e-9960-96ef091aafb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:05.362637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4d206-1cf2-407e-9960-96ef091aafb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:05.363086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4d206-1cf2-407e-9960-96ef091aafb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:05.364646 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4d206-1cf2-407e-9960-96ef091aafb0 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:05.373658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 70/147] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:05.379316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d41b4-30f9-4ccd-83b7-56877882dedf None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:05.406771 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:05.494013 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d41b4-30f9-4ccd-83b7-56877882dedf None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:05.531052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbaeb7f-9caa-4366-b377-fb0686147ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:05.531978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbaeb7f-9caa-4366-b377-fb0686147ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:05.532402 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbaeb7f-9caa-4366-b377-fb0686147ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:05.532821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbaeb7f-9caa-4366-b377-fb0686147ed1 None admin] PATH_INFO: `/v3/users/6c5badd8f28a4d749446523e9d5a6ed3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:05.534165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbaeb7f-9caa-4366-b377-fb0686147ed1 None admin] RBAC: Authorizing `identity:get_user(user_id=6c5badd8f28a4d749446523e9d5a6ed3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:05.535854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbaeb7f-9caa-4366-b377-fb0686147ed1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:05.538598 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 78/148] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:05 2022] GET /identity/v3/users/6c5badd8f28a4d749446523e9d5a6ed3 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:05.543394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4d206-1cf2-407e-9960-96ef091aafb0 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:05.579312 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:05.657800 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4d206-1cf2-407e-9960-96ef091aafb0 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:05.693651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ba9c8-b67f-4f5f-be8a-d3d9858b794b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:05.694561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ba9c8-b67f-4f5f-be8a-d3d9858b794b None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:05.695015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ba9c8-b67f-4f5f-be8a-d3d9858b794b None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:05.695437 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ba9c8-b67f-4f5f-be8a-d3d9858b794b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:05.768179 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ba9c8-b67f-4f5f-be8a-d3d9858b794b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:05.769888 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71ba9c8-b67f-4f5f-be8a-d3d9858b794b None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:05.781279 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbaeb7f-9caa-4366-b377-fb0686147ed1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:05.832010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:05.923496 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 71/149] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:19:05.932003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ba9c8-b67f-4f5f-be8a-d3d9858b794b None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:05.973261 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:05.991817 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbaeb7f-9caa-4366-b377-fb0686147ed1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:06.048907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c025b4-dd54-431a-b004-649dc0a1e537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:06.049563 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c025b4-dd54-431a-b004-649dc0a1e537 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:06.049847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c025b4-dd54-431a-b004-649dc0a1e537 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:06.050123 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c025b4-dd54-431a-b004-649dc0a1e537 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:06.070643 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ba9c8-b67f-4f5f-be8a-d3d9858b794b None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:06.072794 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c025b4-dd54-431a-b004-649dc0a1e537 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:06.077950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-39c025b4-dd54-431a-b004-649dc0a1e537 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:06.077950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:06.077950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:06.077950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:06.077950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:06.077950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:06.077950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:06.077950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:06.077950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:06.077950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:06.077950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:06.077950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:06.077950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:06.077950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:06.077950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:06.077950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:06.078821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:06.078821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:06.078821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:06.078821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:06.078821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:06.078821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:06.078821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:06.078821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:06.078821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:06.078821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:06.078821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:06.078821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:06.078821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:06.078821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:06.078821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:06.078821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:06.079576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:06.079576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:06.079576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:06.079576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:06.079576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:06.079576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:06.079576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:06.079576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:06.079576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:06.079576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:06.079576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:06.079576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:06.079576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 79/150] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:06.086206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c025b4-dd54-431a-b004-649dc0a1e537 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:06.110210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51667038-ce71-4090-b413-fd834dca972b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:06.110873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51667038-ce71-4090-b413-fd834dca972b None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:06.111164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51667038-ce71-4090-b413-fd834dca972b None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:06.111460 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51667038-ce71-4090-b413-fd834dca972b None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:06.112716 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51667038-ce71-4090-b413-fd834dca972b None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:06.113179 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:06.117066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51667038-ce71-4090-b413-fd834dca972b None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:06.118998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 72/151] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:05 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:06.123688 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a9387-7de5-419a-9d9f-bd8fb2351dd2 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:06.124015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a9387-7de5-419a-9d9f-bd8fb2351dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:06.124316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a9387-7de5-419a-9d9f-bd8fb2351dd2 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:06.125744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 73/152] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:06.129674 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a9387-7de5-419a-9d9f-bd8fb2351dd2 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:06.247454 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c025b4-dd54-431a-b004-649dc0a1e537 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:06.261766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a9387-7de5-419a-9d9f-bd8fb2351dd2 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:06.289249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c6733-6e0b-45a0-aa92-547e5d0cd70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:06.289869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c6733-6e0b-45a0-aa92-547e5d0cd70e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:06.290151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c6733-6e0b-45a0-aa92-547e5d0cd70e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:06.290423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c6733-6e0b-45a0-aa92-547e5d0cd70e None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:06.291095 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c6733-6e0b-45a0-aa92-547e5d0cd70e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:06.291400 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c6733-6e0b-45a0-aa92-547e5d0cd70e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:06.293359 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c6733-6e0b-45a0-aa92-547e5d0cd70e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:06.309145 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 80/153] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:06.310894 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f645bdd-1f23-42d6-b596-ac8dbe3b6b48 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:06.311379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f645bdd-1f23-42d6-b596-ac8dbe3b6b48 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:06.311889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f645bdd-1f23-42d6-b596-ac8dbe3b6b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:06.342614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948e897-e214-48fa-b674-4a949a6429d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:06.343242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948e897-e214-48fa-b674-4a949a6429d3 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:06.343524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948e897-e214-48fa-b674-4a949a6429d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:06.343827 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948e897-e214-48fa-b674-4a949a6429d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:06.344579 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948e897-e214-48fa-b674-4a949a6429d3 None admin] RBAC: Adding query filter params (role.id=156b5bb1275a4dbeb61dbdade2c20d21, scope.project.id=c480d76c537348a7a86ae331f79d5e39, user.id=6c5badd8f28a4d749446523e9d5a6ed3) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:06.344696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0f645bdd-1f23-42d6-b596-ac8dbe3b6b48 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:06.344883 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948e897-e214-48fa-b674-4a949a6429d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:06.346285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0f645bdd-1f23-42d6-b596-ac8dbe3b6b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:06.346387 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948e897-e214-48fa-b674-4a949a6429d3 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:06.350579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:06.360218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 74/154] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:06 2022] GET /identity/v3/role_assignments?user.id=6c5badd8f28a4d749446523e9d5a6ed3&scope.project.id=c480d76c537348a7a86ae331f79d5e39&role.id=156b5bb1275a4dbeb61dbdade2c20d21 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:06.361073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948e897-e214-48fa-b674-4a949a6429d3 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:06.474773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:06.488096 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 81/155] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:06.489406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c5c33-2898-4d45-b062-8056e674ccce None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:06.489693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c5c33-2898-4d45-b062-8056e674ccce None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:06.489977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c5c33-2898-4d45-b062-8056e674ccce None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:06.491434 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 82/156] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:06.501363 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f58987-0945-4bbe-a912-53b9ee9e2b64 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:06.501631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f58987-0945-4bbe-a912-53b9ee9e2b64 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:06.501836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f58987-0945-4bbe-a912-53b9ee9e2b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:06.540200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948e897-e214-48fa-b674-4a949a6429d3 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:06.543377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-76f58987-0945-4bbe-a912-53b9ee9e2b64 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:06.545362 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-76f58987-0945-4bbe-a912-53b9ee9e2b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:06.607657 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841d3da-f9e1-47ab-9cd0-ab630d111052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:06.608325 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841d3da-f9e1-47ab-9cd0-ab630d111052 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:06.608609 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841d3da-f9e1-47ab-9cd0-ab630d111052 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:06.608887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0841d3da-f9e1-47ab-9cd0-ab630d111052 None admin] PATH_INFO: `/v3/users/fc82f935e3a047f58c93d87ad0d07ecc` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:06.611405 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841d3da-f9e1-47ab-9cd0-ab630d111052 None admin] RBAC: Authorizing `identity:get_user(user_id=fc82f935e3a047f58c93d87ad0d07ecc)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:06.613679 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0841d3da-f9e1-47ab-9cd0-ab630d111052 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:06.615811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 75/157] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:06 2022] GET /identity/v3/users/fc82f935e3a047f58c93d87ad0d07ecc => generated 289 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:06.620910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841d3da-f9e1-47ab-9cd0-ab630d111052 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:06.700364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 83/158] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:06.787842 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0841d3da-f9e1-47ab-9cd0-ab630d111052 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:06.838305 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:06.848222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a41761-b5c7-4867-886c-14f9a370d152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:06.848883 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a41761-b5c7-4867-886c-14f9a370d152 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:06.849225 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a41761-b5c7-4867-886c-14f9a370d152 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:06.849722 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a41761-b5c7-4867-886c-14f9a370d152 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:06.851166 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a41761-b5c7-4867-886c-14f9a370d152 None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:06.852665 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a41761-b5c7-4867-886c-14f9a370d152 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:06.854718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 76/159] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:06 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:06.860969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f58987-0945-4bbe-a912-53b9ee9e2b64 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:06.948056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:06.989130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f58987-0945-4bbe-a912-53b9ee9e2b64 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:07.025712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15339aa-ef76-4746-803d-d23e15b2fb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:07.026381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15339aa-ef76-4746-803d-d23e15b2fb54 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:07.026665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15339aa-ef76-4746-803d-d23e15b2fb54 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:07.026942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15339aa-ef76-4746-803d-d23e15b2fb54 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39/users/fc82f935e3a047f58c93d87ad0d07ecc/roles/0f1253d845894622b8091c95dfd079d3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:07.032915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15339aa-ef76-4746-803d-d23e15b2fb54 None admin] RBAC: Authorizing `identity:create_grant(project_id=c480d76c537348a7a86ae331f79d5e39, user_id=fc82f935e3a047f58c93d87ad0d07ecc, role_id=0f1253d845894622b8091c95dfd079d3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:07.034389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15339aa-ef76-4746-803d-d23e15b2fb54 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:07.060985 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 84/160] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:06 2022] PUT /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39/users/fc82f935e3a047f58c93d87ad0d07ecc/roles/0f1253d845894622b8091c95dfd079d3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:07.151314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a41761-b5c7-4867-886c-14f9a370d152 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:07.237676 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15339aa-ef76-4746-803d-d23e15b2fb54 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:07.284106 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a41761-b5c7-4867-886c-14f9a370d152 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:07.327646 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba88986-0f09-4ae9-9657-fbb6aaa08cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:07.328211 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba88986-0f09-4ae9-9657-fbb6aaa08cdc None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:07.328487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba88986-0f09-4ae9-9657-fbb6aaa08cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:07.328766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba88986-0f09-4ae9-9657-fbb6aaa08cdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:07.342408 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba88986-0f09-4ae9-9657-fbb6aaa08cdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:07.348517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-aba88986-0f09-4ae9-9657-fbb6aaa08cdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:07.348517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:07.348517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:07.348517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:07.348517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:07.348517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:07.348517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:07.348517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:07.348517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:07.348517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:07.348517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:07.348517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:07.348517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:07.348517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:07.348517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:07.348517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:07.349442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:07.349442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:07.349442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:07.349442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:07.349442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:07.349442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:07.349442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:07.349442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:07.349442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:07.349442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:07.349442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:07.349442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:07.349442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:07.349442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:07.349442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:07.349442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:07.350222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:07.350222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:07.350222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:07.350222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:07.350222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:07.350222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:07.350222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:07.350222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:07.350222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:07.350222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:07.350222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:07.350222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:07.350222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 77/161] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:07.358441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba88986-0f09-4ae9-9657-fbb6aaa08cdc None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:07.436143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15339aa-ef76-4746-803d-d23e15b2fb54 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:07.491557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:07.493625 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:07.502406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294551c5-c76d-4aa3-b6b8-d864ec939bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:07.503068 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294551c5-c76d-4aa3-b6b8-d864ec939bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:07.503351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294551c5-c76d-4aa3-b6b8-d864ec939bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:07.503657 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294551c5-c76d-4aa3-b6b8-d864ec939bf8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:07.524566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294551c5-c76d-4aa3-b6b8-d864ec939bf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:07.530276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-294551c5-c76d-4aa3-b6b8-d864ec939bf8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:07.530276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:07.530276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:07.530276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:07.530276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:07.530276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:07.530276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:07.530276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:07.530276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:07.530276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:07.530276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:07.530276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:07.530276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:07.530276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:07.530276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:07.530276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:07.531133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:07.531133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:07.531133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:07.531133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:07.531133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:07.531133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:07.531133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:07.531133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:07.531133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:07.531133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:07.531133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:07.531133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:07.531133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:07.531133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:07.531133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:07.531133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:07.532032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:07.532032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:07.532032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:07.532032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:07.532032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:07.532032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:07.532032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:07.532032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:07.532032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:07.532032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:07.532032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:07.532032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:07.532032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 85/162] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:07.537459 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ee96f-03f7-4b11-be55-a9e325f26a7f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:07.542322 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ee96f-03f7-4b11-be55-a9e325f26a7f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:07.542606 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ee96f-03f7-4b11-be55-a9e325f26a7f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:07.544764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 86/163] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:07 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:07.545564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ee96f-03f7-4b11-be55-a9e325f26a7f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:07.605885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba88986-0f09-4ae9-9657-fbb6aaa08cdc None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:07.694175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d82e9-bb76-494b-9814-142771bbddc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:07.694794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d82e9-bb76-494b-9814-142771bbddc4 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:07.695078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d82e9-bb76-494b-9814-142771bbddc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:07.695356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d82e9-bb76-494b-9814-142771bbddc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:07.696051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d82e9-bb76-494b-9814-142771bbddc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:07.696356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d82e9-bb76-494b-9814-142771bbddc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:07.697734 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d82e9-bb76-494b-9814-142771bbddc4 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:07.717884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 78/164] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:07.719131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81756772-3d9b-4a21-a027-64fa2efc34b3 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:07.719418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81756772-3d9b-4a21-a027-64fa2efc34b3 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:07.719715 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81756772-3d9b-4a21-a027-64fa2efc34b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:07.748134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ee96f-03f7-4b11-be55-a9e325f26a7f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:07.756620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-81756772-3d9b-4a21-a027-64fa2efc34b3 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:07.757832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-81756772-3d9b-4a21-a027-64fa2efc34b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:07.811880 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c548cf-d738-4b56-89ad-a6d280e76b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:07.812499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c548cf-d738-4b56-89ad-a6d280e76b97 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:07.812778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c548cf-d738-4b56-89ad-a6d280e76b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:07.813049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c548cf-d738-4b56-89ad-a6d280e76b97 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:07.813773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c548cf-d738-4b56-89ad-a6d280e76b97 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:07.814075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c548cf-d738-4b56-89ad-a6d280e76b97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:07.815473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c548cf-d738-4b56-89ad-a6d280e76b97 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:07.834548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 87/165] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:07.835499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c548cf-d738-4b56-89ad-a6d280e76b97 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:07.940329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:07.962499 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 79/166] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:07.964384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81756772-3d9b-4a21-a027-64fa2efc34b3 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:08.013413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c548cf-d738-4b56-89ad-a6d280e76b97 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:08.073283 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bf026-619b-45b5-a832-ff068ae5bb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:08.074004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bf026-619b-45b5-a832-ff068ae5bb6a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:08.075037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bf026-619b-45b5-a832-ff068ae5bb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:08.075314 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bf026-619b-45b5-a832-ff068ae5bb6a None admin] PATH_INFO: `/v3/users/ec5a3f02319e4ee5bdd1b0a6b24084c2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:08.080307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432bf026-619b-45b5-a832-ff068ae5bb6a None admin] RBAC: Authorizing `identity:get_user(user_id=ec5a3f02319e4ee5bdd1b0a6b24084c2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:08.081796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432bf026-619b-45b5-a832-ff068ae5bb6a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:08.083433 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 88/167] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:07 2022] GET /identity/v3/users/ec5a3f02319e4ee5bdd1b0a6b24084c2 => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:08.083795 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:08.090222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bf026-619b-45b5-a832-ff068ae5bb6a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:08.211348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81756772-3d9b-4a21-a027-64fa2efc34b3 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:08.222743 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bf026-619b-45b5-a832-ff068ae5bb6a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:08.244291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:08.276362 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788792d2-6a90-484b-9fbd-ae2be9cb3fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:08.276998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788792d2-6a90-484b-9fbd-ae2be9cb3fde None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:08.277354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788792d2-6a90-484b-9fbd-ae2be9cb3fde None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:08.277621 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788792d2-6a90-484b-9fbd-ae2be9cb3fde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:08.291476 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bd5a4-316c-41c0-a2db-53cf19a0d14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:08.292222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bd5a4-316c-41c0-a2db-53cf19a0d14f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:08.292500 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bd5a4-316c-41c0-a2db-53cf19a0d14f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:08.293311 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bd5a4-316c-41c0-a2db-53cf19a0d14f None admin] PATH_INFO: `/v3/users/6c5badd8f28a4d749446523e9d5a6ed3` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:08.294661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721bd5a4-316c-41c0-a2db-53cf19a0d14f None admin] RBAC: Authorizing `identity:get_user(user_id=6c5badd8f28a4d749446523e9d5a6ed3)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:08.300265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721bd5a4-316c-41c0-a2db-53cf19a0d14f None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:08.302418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 80/168] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:07 2022] GET /identity/v3/users/6c5badd8f28a4d749446523e9d5a6ed3 => generated 275 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:08.308074 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bd5a4-316c-41c0-a2db-53cf19a0d14f None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:08.392002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788792d2-6a90-484b-9fbd-ae2be9cb3fde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:08.394106 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788792d2-6a90-484b-9fbd-ae2be9cb3fde None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:08.537209 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 89/169] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:19:08.538478 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788792d2-6a90-484b-9fbd-ae2be9cb3fde None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:08.567860 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:08.645642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721bd5a4-316c-41c0-a2db-53cf19a0d14f None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:08.659815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788792d2-6a90-484b-9fbd-ae2be9cb3fde None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:08.700194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52cd69-3b16-457c-a1d8-51897ff81e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:08.700494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:08.703472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52cd69-3b16-457c-a1d8-51897ff81e90 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:08.703788 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52cd69-3b16-457c-a1d8-51897ff81e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:08.704302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52cd69-3b16-457c-a1d8-51897ff81e90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:08.727151 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafc464f-616d-4fb6-87d4-bc6e1daac8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:08.727791 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc464f-616d-4fb6-87d4-bc6e1daac8fe None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:08.728069 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc464f-616d-4fb6-87d4-bc6e1daac8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:08.728334 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafc464f-616d-4fb6-87d4-bc6e1daac8fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:08.729406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52cd69-3b16-457c-a1d8-51897ff81e90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:08.736796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-be52cd69-3b16-457c-a1d8-51897ff81e90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:08.736796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:08.736796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:08.736796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:08.736796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:08.736796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:08.736796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:08.736796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:08.736796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:08.736796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:08.736796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:08.736796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:08.736796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:08.736796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:08.736796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:08.736796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:08.737767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:08.737767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:08.737767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:08.737767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:08.737767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:08.737767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:08.737767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:08.737767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:08.737767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:08.737767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:08.737767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:08.737767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:08.737767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:08.737767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:08.737767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:08.737767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:08.738586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:08.738586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:08.738586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:08.738586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:08.738586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:08.738586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:08.738586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:08.738586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:08.738586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:08.738586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:08.738586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:08.738586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:08.738586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 90/170] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:08.740719 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52cd69-3b16-457c-a1d8-51897ff81e90 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:08.857239 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc464f-616d-4fb6-87d4-bc6e1daac8fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:08.858949 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafc464f-616d-4fb6-87d4-bc6e1daac8fe None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:08.889599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:08.928582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52cd69-3b16-457c-a1d8-51897ff81e90 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:08.986312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f4626-cae5-4bbf-8d81-d8271a5174f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:08.986954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f4626-cae5-4bbf-8d81-d8271a5174f3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:08.987232 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f4626-cae5-4bbf-8d81-d8271a5174f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:08.988398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f4626-cae5-4bbf-8d81-d8271a5174f3 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:08.989588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f4626-cae5-4bbf-8d81-d8271a5174f3 None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:08.991075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f4626-cae5-4bbf-8d81-d8271a5174f3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:08.992840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 91/171] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:08 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:08.999350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f4626-cae5-4bbf-8d81-d8271a5174f3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:09.053547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 81/172] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 746 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:19:09.059656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb41d8-4a0b-436b-bde4-66ceae57410e None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.059656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb41d8-4a0b-436b-bde4-66ceae57410e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.059656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb41d8-4a0b-436b-bde4-66ceae57410e None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.059656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 82/173] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:09.060834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb41d8-4a0b-436b-bde4-66ceae57410e None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:09.100484 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:09.153057 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f4626-cae5-4bbf-8d81-d8271a5174f3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:09.190002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b3452-07ae-468a-8af7-422bbd2fe9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:09.190719 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b3452-07ae-468a-8af7-422bbd2fe9d8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.191102 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b3452-07ae-468a-8af7-422bbd2fe9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.191379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b3452-07ae-468a-8af7-422bbd2fe9d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.192075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb41d8-4a0b-436b-bde4-66ceae57410e None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:09.192195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b3452-07ae-468a-8af7-422bbd2fe9d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:09.192440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b3452-07ae-468a-8af7-422bbd2fe9d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:09.193952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b3452-07ae-468a-8af7-422bbd2fe9d8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:09.203488 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 92/174] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:09.204509 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b3452-07ae-468a-8af7-422bbd2fe9d8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:09.218555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:09.240256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c3f67-d711-497a-93b4-782cf66a082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:09.240868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c3f67-d711-497a-93b4-782cf66a082c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.241206 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c3f67-d711-497a-93b4-782cf66a082c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.241480 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c3f67-d711-497a-93b4-782cf66a082c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.242320 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c3f67-d711-497a-93b4-782cf66a082c None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=c480d76c537348a7a86ae331f79d5e39, user.id=ec5a3f02319e4ee5bdd1b0a6b24084c2) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:09.242738 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c3f67-d711-497a-93b4-782cf66a082c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:09.244372 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c3f67-d711-497a-93b4-782cf66a082c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:09.251199 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 83/175] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:09 2022] GET /identity/v3/role_assignments?user.id=ec5a3f02319e4ee5bdd1b0a6b24084c2&scope.project.id=c480d76c537348a7a86ae331f79d5e39&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:09.256691 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20672a6-60fb-4607-a6e7-a2fcaa2ae76e None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.256980 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20672a6-60fb-4607-a6e7-a2fcaa2ae76e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.257259 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20672a6-60fb-4607-a6e7-a2fcaa2ae76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.291774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-d20672a6-60fb-4607-a6e7-a2fcaa2ae76e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:09.293953 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d20672a6-60fb-4607-a6e7-a2fcaa2ae76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:09.318038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b3452-07ae-468a-8af7-422bbd2fe9d8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:09.357984 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de444a7a-d167-469b-bce7-cccd2e57afb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:09.358642 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de444a7a-d167-469b-bce7-cccd2e57afb5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.358922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de444a7a-d167-469b-bce7-cccd2e57afb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.359201 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de444a7a-d167-469b-bce7-cccd2e57afb5 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.360537 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de444a7a-d167-469b-bce7-cccd2e57afb5 None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:09.362039 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de444a7a-d167-469b-bce7-cccd2e57afb5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:09.364018 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 93/176] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:09 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:09.370377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2136a-3d74-4f0e-a32a-c28a46f26684 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.370687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2136a-3d74-4f0e-a32a-c28a46f26684 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.371511 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2136a-3d74-4f0e-a32a-c28a46f26684 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.373128 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 94/177] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:09.374257 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2136a-3d74-4f0e-a32a-c28a46f26684 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:09.386042 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 84/178] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:09.387439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd2a89-7584-40d7-ae3f-1987f6df9767 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.387779 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd2a89-7584-40d7-ae3f-1987f6df9767 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.388115 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd2a89-7584-40d7-ae3f-1987f6df9767 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.389727 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 85/179] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:09.390461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd2a89-7584-40d7-ae3f-1987f6df9767 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:09.410537 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:09.472578 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:09.506110 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2136a-3d74-4f0e-a32a-c28a46f26684 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:09.550694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3043bd-4eaa-44a6-a0c1-6697d10714d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:09.551450 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3043bd-4eaa-44a6-a0c1-6697d10714d5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.551915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3043bd-4eaa-44a6-a0c1-6697d10714d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.552197 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3043bd-4eaa-44a6-a0c1-6697d10714d5 None admin] PATH_INFO: `/v3/users/6c5badd8f28a4d749446523e9d5a6ed3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.553657 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3043bd-4eaa-44a6-a0c1-6697d10714d5 None admin] RBAC: Authorizing `identity:get_user(user_id=6c5badd8f28a4d749446523e9d5a6ed3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:09.554823 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3043bd-4eaa-44a6-a0c1-6697d10714d5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:09.557509 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 95/180] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:09 2022] GET /identity/v3/users/6c5badd8f28a4d749446523e9d5a6ed3 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:09.558753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc23e1b-5996-4c00-9191-59d224434c1e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.563905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc23e1b-5996-4c00-9191-59d224434c1e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.564201 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc23e1b-5996-4c00-9191-59d224434c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.574865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd2a89-7584-40d7-ae3f-1987f6df9767 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:09.585167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ffc23e1b-5996-4c00-9191-59d224434c1e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:09.586480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ffc23e1b-5996-4c00-9191-59d224434c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:09.621468 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1280b93-c940-4277-95ef-be75b673b653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:09.622448 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1280b93-c940-4277-95ef-be75b673b653 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.622962 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1280b93-c940-4277-95ef-be75b673b653 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.623727 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1280b93-c940-4277-95ef-be75b673b653 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.625129 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1280b93-c940-4277-95ef-be75b673b653 None admin] RBAC: Adding query filter params (role.id=4e002a22056744acb8c40a3236cab404, scope.project.id=c480d76c537348a7a86ae331f79d5e39, user.id=6c5badd8f28a4d749446523e9d5a6ed3) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:09.625873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1280b93-c940-4277-95ef-be75b673b653 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:09.628258 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1280b93-c940-4277-95ef-be75b673b653 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:09.635521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 86/181] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:09 2022] GET /identity/v3/role_assignments?user.id=6c5badd8f28a4d749446523e9d5a6ed3&scope.project.id=c480d76c537348a7a86ae331f79d5e39&role.id=4e002a22056744acb8c40a3236cab404 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:09.636913 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da46be8-6088-4252-bde7-58f9ef1ae5b7 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.637271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da46be8-6088-4252-bde7-58f9ef1ae5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.637462 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da46be8-6088-4252-bde7-58f9ef1ae5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.676863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-6da46be8-6088-4252-bde7-58f9ef1ae5b7 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:09.678008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-6da46be8-6088-4252-bde7-58f9ef1ae5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:09.681008 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 96/182] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:09.683245 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc23e1b-5996-4c00-9191-59d224434c1e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:09.798021 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:09.826459 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc23e1b-5996-4c00-9191-59d224434c1e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:09.838038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 87/183] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:09.842541 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da46be8-6088-4252-bde7-58f9ef1ae5b7 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:09.852086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:09.904338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a98c5ca-491a-4616-9d28-2c4e98ebcca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:09.904965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98c5ca-491a-4616-9d28-2c4e98ebcca8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:09.905246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98c5ca-491a-4616-9d28-2c4e98ebcca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:09.905520 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98c5ca-491a-4616-9d28-2c4e98ebcca8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:09.971187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da46be8-6088-4252-bde7-58f9ef1ae5b7 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.013946 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e20c3e-63cf-4807-9c0e-c4c67d7f6138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.014808 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e20c3e-63cf-4807-9c0e-c4c67d7f6138 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.015157 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e20c3e-63cf-4807-9c0e-c4c67d7f6138 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.015485 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e20c3e-63cf-4807-9c0e-c4c67d7f6138 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.026914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e20c3e-63cf-4807-9c0e-c4c67d7f6138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:10.028792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a98c5ca-491a-4616-9d28-2c4e98ebcca8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.030267 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a98c5ca-491a-4616-9d28-2c4e98ebcca8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:10.032740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-c8e20c3e-63cf-4807-9c0e-c4c67d7f6138 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:10.032740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:10.032740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:10.032740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:10.032740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:10.032740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:10.032740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:10.032740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:10.032740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:10.032740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:10.032740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:10.032740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:10.032740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:10.032740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:10.032740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:10.032740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:10.033615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:10.033615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:10.033615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:10.033615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:10.033615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:10.033615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:10.033615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:10.033615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:10.033615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:10.033615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:10.033615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:10.033615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:10.033615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:10.033615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:10.033615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:10.033615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:10.034423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:10.034423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:10.034423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:10.034423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:10.034423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:10.034423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:10.034423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:10.034423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:10.034423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:10.034423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:10.034423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:10.034423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:10.034423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 88/184] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:10.040919 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e20c3e-63cf-4807-9c0e-c4c67d7f6138 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.123784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:10.126137 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:10.151852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 97/185] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:10.154080 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef41d46-b6b3-4c26-883f-f49e770b5859 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.154652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef41d46-b6b3-4c26-883f-f49e770b5859 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.154652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef41d46-b6b3-4c26-883f-f49e770b5859 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.155901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 98/186] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:10.156652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef41d46-b6b3-4c26-883f-f49e770b5859 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.173215 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e20c3e-63cf-4807-9c0e-c4c67d7f6138 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.209239 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8aaa6-8bfe-4362-80ec-0624ce949f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.209848 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8aaa6-8bfe-4362-80ec-0624ce949f2c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.210130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8aaa6-8bfe-4362-80ec-0624ce949f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.210403 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8aaa6-8bfe-4362-80ec-0624ce949f2c None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.211068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8aaa6-8bfe-4362-80ec-0624ce949f2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:10.211374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8aaa6-8bfe-4362-80ec-0624ce949f2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.212806 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8aaa6-8bfe-4362-80ec-0624ce949f2c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:10.221684 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 89/187] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:10.226721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8aaa6-8bfe-4362-80ec-0624ce949f2c None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.261120 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef41d46-b6b3-4c26-883f-f49e770b5859 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.294727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1118dc4-f348-43a4-aa9d-af5e61f3e2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.295347 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1118dc4-f348-43a4-aa9d-af5e61f3e2db None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.295649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1118dc4-f348-43a4-aa9d-af5e61f3e2db None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.295919 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1118dc4-f348-43a4-aa9d-af5e61f3e2db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.305668 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1118dc4-f348-43a4-aa9d-af5e61f3e2db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:10.310712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a1118dc4-f348-43a4-aa9d-af5e61f3e2db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:10.310712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:10.310712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:10.310712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:10.310712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:10.310712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:10.310712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:10.310712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:10.310712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:10.310712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:10.310712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:10.310712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:10.310712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:10.310712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:10.310712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:10.310712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:10.311727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:10.311727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:10.311727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:10.311727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:10.311727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:10.311727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:10.311727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:10.311727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:10.311727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:10.311727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:10.311727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:10.311727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:10.311727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:10.311727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:10.311727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:10.311727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:10.312542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:10.312542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:10.312542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:10.312542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:10.312542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:10.312542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:10.312542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:10.312542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:10.312542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:10.312542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:10.312542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:10.312542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:10.312542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 99/188] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:10.313226 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b13c2-8d60-4160-84b3-19fe616e4ad2 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.313347 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b13c2-8d60-4160-84b3-19fe616e4ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.313711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b13c2-8d60-4160-84b3-19fe616e4ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.329986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8aaa6-8bfe-4362-80ec-0624ce949f2c None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.330928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-523b13c2-8d60-4160-84b3-19fe616e4ad2 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:10.332145 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-523b13c2-8d60-4160-84b3-19fe616e4ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:10.356662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:10.362339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:10.365085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b4fe6-9901-4b96-a86f-e82e92f1409a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.365717 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b4fe6-9901-4b96-a86f-e82e92f1409a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.365993 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b4fe6-9901-4b96-a86f-e82e92f1409a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.366262 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b4fe6-9901-4b96-a86f-e82e92f1409a None admin] PATH_INFO: `/v3/projects/5f4711b51fa54184981937491c080b51` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.367347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b4fe6-9901-4b96-a86f-e82e92f1409a None admin] RBAC: Authorizing `identity:get_project(project_id=5f4711b51fa54184981937491c080b51)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.368820 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b4fe6-9901-4b96-a86f-e82e92f1409a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:10.370442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 90/189] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/projects/5f4711b51fa54184981937491c080b51 => generated 312 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:10.371238 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b4fe6-9901-4b96-a86f-e82e92f1409a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.401452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 100/190] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:10.415815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b13c2-8d60-4160-84b3-19fe616e4ad2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.492152 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b4fe6-9901-4b96-a86f-e82e92f1409a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.512233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b13c2-8d60-4160-84b3-19fe616e4ad2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.525359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd95d6-7c51-4603-bf63-f76a84de0428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.525993 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd95d6-7c51-4603-bf63-f76a84de0428 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.526285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd95d6-7c51-4603-bf63-f76a84de0428 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.526558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd95d6-7c51-4603-bf63-f76a84de0428 None admin] PATH_INFO: `/v3/users/d39bf2c0c3d24386aa925ba8ecbf4f2c` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.542252 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd95d6-7c51-4603-bf63-f76a84de0428 None admin] RBAC: Authorizing `identity:get_user(user_id=d39bf2c0c3d24386aa925ba8ecbf4f2c)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.544312 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd95d6-7c51-4603-bf63-f76a84de0428 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:10.545982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 91/191] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/users/d39bf2c0c3d24386aa925ba8ecbf4f2c => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:10.546754 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd95d6-7c51-4603-bf63-f76a84de0428 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.554645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ca769-61ad-4dcf-ab9c-1d2f5f022e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.555591 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ca769-61ad-4dcf-ab9c-1d2f5f022e77 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.556035 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ca769-61ad-4dcf-ab9c-1d2f5f022e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.556455 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ca769-61ad-4dcf-ab9c-1d2f5f022e77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.557900 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ca769-61ad-4dcf-ab9c-1d2f5f022e77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.559996 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ca769-61ad-4dcf-ab9c-1d2f5f022e77 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:10.561999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 101/192] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:10.562942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ca769-61ad-4dcf-ab9c-1d2f5f022e77 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.604100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:10.607558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:10.668161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd95d6-7c51-4603-bf63-f76a84de0428 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.675454 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ca769-61ad-4dcf-ab9c-1d2f5f022e77 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.702663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c2461-981a-4bdc-a9da-e507c5dad09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.703272 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c2461-981a-4bdc-a9da-e507c5dad09a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.703551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c2461-981a-4bdc-a9da-e507c5dad09a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.703855 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c2461-981a-4bdc-a9da-e507c5dad09a None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.704522 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573c2461-981a-4bdc-a9da-e507c5dad09a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:10.704828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573c2461-981a-4bdc-a9da-e507c5dad09a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.706211 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573c2461-981a-4bdc-a9da-e507c5dad09a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:10.710104 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a64ca9-ccff-4b4e-99ed-c9315578863d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.710699 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a64ca9-ccff-4b4e-99ed-c9315578863d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.710988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a64ca9-ccff-4b4e-99ed-c9315578863d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.711266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a64ca9-ccff-4b4e-99ed-c9315578863d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.712029 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a64ca9-ccff-4b4e-99ed-c9315578863d None admin] RBAC: Adding query filter params (role.id=156b5bb1275a4dbeb61dbdade2c20d21, scope.project.id=5f4711b51fa54184981937491c080b51, user.id=6c5badd8f28a4d749446523e9d5a6ed3) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:10.712343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a64ca9-ccff-4b4e-99ed-c9315578863d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.713776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a64ca9-ccff-4b4e-99ed-c9315578863d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:10.714045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 92/193] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:10.715415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c2461-981a-4bdc-a9da-e507c5dad09a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.720351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 102/194] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/role_assignments?user.id=6c5badd8f28a4d749446523e9d5a6ed3&scope.project.id=5f4711b51fa54184981937491c080b51&role.id=156b5bb1275a4dbeb61dbdade2c20d21 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:10.721005 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a64ca9-ccff-4b4e-99ed-c9315578863d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.837039 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:10.837860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c2461-981a-4bdc-a9da-e507c5dad09a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.842088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a64ca9-ccff-4b4e-99ed-c9315578863d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:10.856346 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:10.889968 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391dc1a-4028-49ca-8be7-95981cbbe91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.890590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391dc1a-4028-49ca-8be7-95981cbbe91d None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.890830 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdd225-18d0-497b-9635-321e5ef2a677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:10.890962 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391dc1a-4028-49ca-8be7-95981cbbe91d None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.891128 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391dc1a-4028-49ca-8be7-95981cbbe91d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.891448 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdd225-18d0-497b-9635-321e5ef2a677 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:10.891759 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdd225-18d0-497b-9635-321e5ef2a677 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:10.892042 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdd225-18d0-497b-9635-321e5ef2a677 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:10.892697 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdd225-18d0-497b-9635-321e5ef2a677 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.894155 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdd225-18d0-497b-9635-321e5ef2a677 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:10.926287 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 103/195] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:10 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:10.927143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdd225-18d0-497b-9635-321e5ef2a677 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:10.991823 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391dc1a-4028-49ca-8be7-95981cbbe91d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:10.993383 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391dc1a-4028-49ca-8be7-95981cbbe91d None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:11.057550 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdd225-18d0-497b-9635-321e5ef2a677 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:11.103773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d94c6-bd26-40b2-9e93-b86200d0a499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:11.104351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d94c6-bd26-40b2-9e93-b86200d0a499 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.104636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d94c6-bd26-40b2-9e93-b86200d0a499 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.104907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d94c6-bd26-40b2-9e93-b86200d0a499 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.106546 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 93/196] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:11.107658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391dc1a-4028-49ca-8be7-95981cbbe91d None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:11.116473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d94c6-bd26-40b2-9e93-b86200d0a499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:11.121869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e76d94c6-bd26-40b2-9e93-b86200d0a499 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:11.121869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:11.121869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:11.121869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:11.121869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:11.121869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:11.121869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:11.121869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:11.121869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:11.121869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:11.121869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:11.121869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:11.121869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:11.121869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:11.121869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:11.121869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:11.122754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:11.122754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:11.122754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:11.122754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:11.122754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:11.122754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:11.122754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:11.122754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:11.122754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:11.122754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:11.122754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:11.122754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:11.122754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:11.122754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:11.122754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:11.122754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:11.123557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:11.123557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:11.123557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:11.123557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:11.123557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:11.123557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:11.123557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:11.123557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:11.123557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:11.123557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:11.123557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:11.123557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:11.123557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 104/197] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:11.124878 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d94c6-bd26-40b2-9e93-b86200d0a499 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:11.131308 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:11.142721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:11.245089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391dc1a-4028-49ca-8be7-95981cbbe91d None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:11.257127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d94c6-bd26-40b2-9e93-b86200d0a499 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:11.293926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581788b6-5359-4b5f-855f-9555f07a86b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:11.294944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581788b6-5359-4b5f-855f-9555f07a86b6 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.295448 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581788b6-5359-4b5f-855f-9555f07a86b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.295939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581788b6-5359-4b5f-855f-9555f07a86b6 None admin] PATH_INFO: `/v3/users/d39bf2c0c3d24386aa925ba8ecbf4f2c` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.297191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b090a7-f239-4c86-a78a-53d3fcc118f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:11.297447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581788b6-5359-4b5f-855f-9555f07a86b6 None admin] RBAC: Authorizing `identity:get_user(user_id=d39bf2c0c3d24386aa925ba8ecbf4f2c)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:11.297835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b090a7-f239-4c86-a78a-53d3fcc118f7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.298118 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b090a7-f239-4c86-a78a-53d3fcc118f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.298394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b090a7-f239-4c86-a78a-53d3fcc118f7 None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.299548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b090a7-f239-4c86-a78a-53d3fcc118f7 None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:11.299843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581788b6-5359-4b5f-855f-9555f07a86b6 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:11.301093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b090a7-f239-4c86-a78a-53d3fcc118f7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:11.301809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 94/198] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:11 2022] GET /identity/v3/users/d39bf2c0c3d24386aa925ba8ecbf4f2c => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:11.302710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581788b6-5359-4b5f-855f-9555f07a86b6 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:11.302809 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 105/199] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:11 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:11.308115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b090a7-f239-4c86-a78a-53d3fcc118f7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:11.400132 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:11.421849 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581788b6-5359-4b5f-855f-9555f07a86b6 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:11.424045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:11.447429 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b090a7-f239-4c86-a78a-53d3fcc118f7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:11.464392 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c0f61-8a5f-4eb1-ad5d-568b6bc1048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:11.465320 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c0f61-8a5f-4eb1-ad5d-568b6bc1048a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.465768 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c0f61-8a5f-4eb1-ad5d-568b6bc1048a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.466187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c0f61-8a5f-4eb1-ad5d-568b6bc1048a None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.466996 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c0f61-8a5f-4eb1-ad5d-568b6bc1048a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:11.467510 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c0f61-8a5f-4eb1-ad5d-568b6bc1048a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:11.469102 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c0f61-8a5f-4eb1-ad5d-568b6bc1048a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:11.478170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 95/200] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:11.479386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c0f61-8a5f-4eb1-ad5d-568b6bc1048a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:11.490334 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8764270-6725-4ba9-ba97-65e73a20a41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:11.491020 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8764270-6725-4ba9-ba97-65e73a20a41f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.491314 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8764270-6725-4ba9-ba97-65e73a20a41f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.491598 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8764270-6725-4ba9-ba97-65e73a20a41f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.611294 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8764270-6725-4ba9-ba97-65e73a20a41f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:11.613324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8764270-6725-4ba9-ba97-65e73a20a41f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:11.659570 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c0f61-8a5f-4eb1-ad5d-568b6bc1048a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:11.726893 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25354dc-f115-4873-92b2-4f58b7fe8b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:11.727962 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25354dc-f115-4873-92b2-4f58b7fe8b77 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:11.728459 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25354dc-f115-4873-92b2-4f58b7fe8b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:11.728928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25354dc-f115-4873-92b2-4f58b7fe8b77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:11.729867 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25354dc-f115-4873-92b2-4f58b7fe8b77 None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=d9b6194f22b34cd695de9f04dace7e99, user.id=d39bf2c0c3d24386aa925ba8ecbf4f2c) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:11.731653 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25354dc-f115-4873-92b2-4f58b7fe8b77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:11.735194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25354dc-f115-4873-92b2-4f58b7fe8b77 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:11.747487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 96/201] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:11 2022] GET /identity/v3/role_assignments?user.id=d39bf2c0c3d24386aa925ba8ecbf4f2c&scope.project.id=d9b6194f22b34cd695de9f04dace7e99&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:11.754696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25354dc-f115-4873-92b2-4f58b7fe8b77 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:11.804571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:11.821120 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:11.850423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 106/202] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:19:11.856478 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8764270-6725-4ba9-ba97-65e73a20a41f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:12.053538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8764270-6725-4ba9-ba97-65e73a20a41f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:12.095648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25354dc-f115-4873-92b2-4f58b7fe8b77 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:12.131633 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6d63b-8553-49bc-91de-481910abb40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:12.132298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6d63b-8553-49bc-91de-481910abb40f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:12.132581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6d63b-8553-49bc-91de-481910abb40f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:12.132852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd6d63b-8553-49bc-91de-481910abb40f None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:12.136481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6d63b-8553-49bc-91de-481910abb40f None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:12.138076 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd6d63b-8553-49bc-91de-481910abb40f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:12.139981 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 107/203] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:11 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:12.146122 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6d63b-8553-49bc-91de-481910abb40f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:12.182442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07eed2-bb17-4aab-a636-3d916c187048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:12.185246 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07eed2-bb17-4aab-a636-3d916c187048 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:12.185533 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07eed2-bb17-4aab-a636-3d916c187048 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:12.185811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07eed2-bb17-4aab-a636-3d916c187048 None admin] PATH_INFO: `/v3/users/fc82f935e3a047f58c93d87ad0d07ecc` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:12.189648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07eed2-bb17-4aab-a636-3d916c187048 None admin] RBAC: Authorizing `identity:get_user(user_id=fc82f935e3a047f58c93d87ad0d07ecc)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:12.191958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07eed2-bb17-4aab-a636-3d916c187048 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:12.194582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 97/204] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:11 2022] GET /identity/v3/users/fc82f935e3a047f58c93d87ad0d07ecc => generated 289 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:12.200940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07eed2-bb17-4aab-a636-3d916c187048 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:12.273954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:12.342423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd6d63b-8553-49bc-91de-481910abb40f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:12.392899 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f675d1-81c1-40fb-82c2-b1ee938d730f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:12.396771 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f675d1-81c1-40fb-82c2-b1ee938d730f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:12.397292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f675d1-81c1-40fb-82c2-b1ee938d730f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:12.397798 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f675d1-81c1-40fb-82c2-b1ee938d730f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:12.398738 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f675d1-81c1-40fb-82c2-b1ee938d730f None admin] RBAC: Adding query filter params (role.id=4e002a22056744acb8c40a3236cab404, scope.project.id=d9b6194f22b34cd695de9f04dace7e99, user.id=d39bf2c0c3d24386aa925ba8ecbf4f2c) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:12.399852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f675d1-81c1-40fb-82c2-b1ee938d730f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:12.401923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f675d1-81c1-40fb-82c2-b1ee938d730f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:12.414254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 108/205] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:12 2022] GET /identity/v3/role_assignments?user.id=d39bf2c0c3d24386aa925ba8ecbf4f2c&scope.project.id=d9b6194f22b34cd695de9f04dace7e99&role.id=4e002a22056744acb8c40a3236cab404 => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:12.414777 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:12.445336 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07eed2-bb17-4aab-a636-3d916c187048 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:12.486946 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba0ad3-6303-4e1d-a72f-807324a95f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:12.487980 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba0ad3-6303-4e1d-a72f-807324a95f06 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:12.488468 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba0ad3-6303-4e1d-a72f-807324a95f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:12.488969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba0ad3-6303-4e1d-a72f-807324a95f06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:12.573692 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba0ad3-6303-4e1d-a72f-807324a95f06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:12.580154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba0ad3-6303-4e1d-a72f-807324a95f06 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:12.680891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:12.707075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 98/206] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:12.713112 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f675d1-81c1-40fb-82c2-b1ee938d730f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:12.862739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f675d1-81c1-40fb-82c2-b1ee938d730f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:12.892272 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:12.915597 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e3d5a-f9b1-4056-86d2-613fff514249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:12.916615 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e3d5a-f9b1-4056-86d2-613fff514249 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:12.917118 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e3d5a-f9b1-4056-86d2-613fff514249 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:12.917614 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e3d5a-f9b1-4056-86d2-613fff514249 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:12.918952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e3d5a-f9b1-4056-86d2-613fff514249 None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:12.920634 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e3d5a-f9b1-4056-86d2-613fff514249 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:12.922560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 109/207] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:12 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:12.928597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba0ad3-6303-4e1d-a72f-807324a95f06 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:13.054963 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba0ad3-6303-4e1d-a72f-807324a95f06 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:13.101770 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0f576-c4dc-4d56-a15b-9281673d6e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:13.102743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0f576-c4dc-4d56-a15b-9281673d6e4a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:13.103248 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0f576-c4dc-4d56-a15b-9281673d6e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:13.103749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0f576-c4dc-4d56-a15b-9281673d6e4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:13.104674 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0f576-c4dc-4d56-a15b-9281673d6e4a None admin] RBAC: Adding query filter params (role.id=0f1253d845894622b8091c95dfd079d3, scope.project.id=c480d76c537348a7a86ae331f79d5e39, user.id=fc82f935e3a047f58c93d87ad0d07ecc) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:13.105209 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0f576-c4dc-4d56-a15b-9281673d6e4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:13.106858 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0f576-c4dc-4d56-a15b-9281673d6e4a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:13.113303 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 99/208] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:12 2022] GET /identity/v3/role_assignments?user.id=fc82f935e3a047f58c93d87ad0d07ecc&scope.project.id=c480d76c537348a7a86ae331f79d5e39&role.id=0f1253d845894622b8091c95dfd079d3 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:13.163824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bd84a-d3ff-443f-95f3-14a4ab6ef7dc None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:13.164472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bd84a-d3ff-443f-95f3-14a4ab6ef7dc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:13.165028 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bd84a-d3ff-443f-95f3-14a4ab6ef7dc None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:13.166629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 110/209] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:13.172813 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355ccaf-d505-4bae-b5a1-99e7cb86a795 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:13.173425 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355ccaf-d505-4bae-b5a1-99e7cb86a795 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:13.173894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4355ccaf-d505-4bae-b5a1-99e7cb86a795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:13.194097 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e36015-47fb-455f-aa97-f7c157ee97f3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:13.194703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e36015-47fb-455f-aa97-f7c157ee97f3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:13.195176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e36015-47fb-455f-aa97-f7c157ee97f3 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:13.196886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 111/210] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:19:13.198207 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-4355ccaf-d505-4bae-b5a1-99e7cb86a795 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:13.200267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-4355ccaf-d505-4bae-b5a1-99e7cb86a795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:13.207865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19d256-27f6-444e-bcf7-f9d82ca58a57 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:13.208187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19d256-27f6-444e-bcf7-f9d82ca58a57 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:13.208422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19d256-27f6-444e-bcf7-f9d82ca58a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:13.236985 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7f19d256-27f6-444e-bcf7-f9d82ca58a57 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:13.238202 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7f19d256-27f6-444e-bcf7-f9d82ca58a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:13.249927 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:13.312189 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 100/211] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:13.369869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 112/212] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:13.610747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355ccaf-d505-4bae-b5a1-99e7cb86a795 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:13.659923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19d256-27f6-444e-bcf7-f9d82ca58a57 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:13.725465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4355ccaf-d505-4bae-b5a1-99e7cb86a795 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:13.766501 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1bdbf4-b22d-4bb4-b4cc-a9422ce56312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:13.767852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bdbf4-b22d-4bb4-b4cc-a9422ce56312 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:13.768348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bdbf4-b22d-4bb4-b4cc-a9422ce56312 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:13.768822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bdbf4-b22d-4bb4-b4cc-a9422ce56312 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:13.772947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1bdbf4-b22d-4bb4-b4cc-a9422ce56312 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:13.774162 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1bdbf4-b22d-4bb4-b4cc-a9422ce56312 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:13.776138 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 101/213] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:13.786863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1bdbf4-b22d-4bb4-b4cc-a9422ce56312 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:13.791397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19d256-27f6-444e-bcf7-f9d82ca58a57 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:13.807014 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:13.830917 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cf1e5-45f7-41cd-80db-5c560a94a5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:13.831427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cf1e5-45f7-41cd-80db-5c560a94a5ed None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:13.831749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cf1e5-45f7-41cd-80db-5c560a94a5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:13.832023 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cf1e5-45f7-41cd-80db-5c560a94a5ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:13.845241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465cf1e5-45f7-41cd-80db-5c560a94a5ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:13.846640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:13.851357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-465cf1e5-45f7-41cd-80db-5c560a94a5ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:13.851357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:13.851357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:13.851357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:13.851357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:13.851357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:13.851357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:13.851357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:13.851357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:13.851357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:13.851357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:13.851357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:13.851357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:13.851357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:13.851357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:13.851357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:13.852318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:13.852318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:13.852318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:13.852318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:13.852318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:13.852318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:13.852318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:13.852318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:13.852318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:13.852318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:13.852318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:13.852318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:13.852318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:13.852318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:13.852318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:13.852318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:13.853133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:13.853133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:13.853133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:13.853133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:13.853133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:13.853133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:13.853133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:13.853133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:13.853133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:13.853133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:13.853133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:13.853133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:13.853133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 113/214] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:13.858978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cf1e5-45f7-41cd-80db-5c560a94a5ed None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:13.930751 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1bdbf4-b22d-4bb4-b4cc-a9422ce56312 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:13.967109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fbacc-2ad6-4f84-855d-cda7e09d2a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:13.967779 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fbacc-2ad6-4f84-855d-cda7e09d2a67 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:13.968066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fbacc-2ad6-4f84-855d-cda7e09d2a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:13.968493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fbacc-2ad6-4f84-855d-cda7e09d2a67 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:13.969150 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fbacc-2ad6-4f84-855d-cda7e09d2a67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:13.970619 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fbacc-2ad6-4f84-855d-cda7e09d2a67 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:13.999673 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 102/215] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:13 2022] POST /identity/v3/users => generated 297 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:14.014991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cf1e5-45f7-41cd-80db-5c560a94a5ed None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:14.060393 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707073c-d345-4ad6-b8e9-2585b12b5cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:14.061417 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707073c-d345-4ad6-b8e9-2585b12b5cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:14.062002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707073c-d345-4ad6-b8e9-2585b12b5cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:14.062499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707073c-d345-4ad6-b8e9-2585b12b5cb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:14.063544 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707073c-d345-4ad6-b8e9-2585b12b5cb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:14.064120 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707073c-d345-4ad6-b8e9-2585b12b5cb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:14.065811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707073c-d345-4ad6-b8e9-2585b12b5cb8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:14.077137 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 114/216] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:14.082525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fbacc-2ad6-4f84-855d-cda7e09d2a67 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:14.212269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:14.212914 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d1263-406f-4f30-ac90-e2a98135ae15 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:14.213277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d1263-406f-4f30-ac90-e2a98135ae15 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:14.213614 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d1263-406f-4f30-ac90-e2a98135ae15 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:14.215776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 115/217] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:14.221146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44245fdd-7975-4d63-8909-cc0dba8b1d27 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:14.221283 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fbacc-2ad6-4f84-855d-cda7e09d2a67 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:14.221467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44245fdd-7975-4d63-8909-cc0dba8b1d27 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:14.221738 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44245fdd-7975-4d63-8909-cc0dba8b1d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:14.248576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-44245fdd-7975-4d63-8909-cc0dba8b1d27 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:14.249535 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-44245fdd-7975-4d63-8909-cc0dba8b1d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:14.263153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c212af-d836-4c1e-adb4-d46c85bf4602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:14.263850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c212af-d836-4c1e-adb4-d46c85bf4602 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:14.264133 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c212af-d836-4c1e-adb4-d46c85bf4602 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:14.264710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c212af-d836-4c1e-adb4-d46c85bf4602 None admin] PATH_INFO: `/v3/users/7f4b926506fe49049f9313f9cd6708cf` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:14.282665 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c212af-d836-4c1e-adb4-d46c85bf4602 None admin] RBAC: Authorizing `identity:get_user(user_id=7f4b926506fe49049f9313f9cd6708cf)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:14.286420 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c212af-d836-4c1e-adb4-d46c85bf4602 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:14.286420 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 103/218] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:14 2022] GET /identity/v3/users/7f4b926506fe49049f9313f9cd6708cf => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:14.290913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:14.291369 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c212af-d836-4c1e-adb4-d46c85bf4602 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:14.356092 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 116/219] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:14.415287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c212af-d836-4c1e-adb4-d46c85bf4602 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:14.452791 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374863a-7048-43c7-ac47-f724111b6b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:14.453421 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374863a-7048-43c7-ac47-f724111b6b6c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:14.453703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374863a-7048-43c7-ac47-f724111b6b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:14.453978 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374863a-7048-43c7-ac47-f724111b6b6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:14.481605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:14.529317 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374863a-7048-43c7-ac47-f724111b6b6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:14.530862 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374863a-7048-43c7-ac47-f724111b6b6c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:14.634827 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 104/220] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:14.641139 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44245fdd-7975-4d63-8909-cc0dba8b1d27 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:14.647195 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374863a-7048-43c7-ac47-f724111b6b6c None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:14.713787 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:14.763739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374863a-7048-43c7-ac47-f724111b6b6c None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:14.831614 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44245fdd-7975-4d63-8909-cc0dba8b1d27 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:14.832636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685898b-b09f-4800-9c94-ebb66a906339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:14.833619 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685898b-b09f-4800-9c94-ebb66a906339 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:14.834170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685898b-b09f-4800-9c94-ebb66a906339 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:14.834753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685898b-b09f-4800-9c94-ebb66a906339 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:14.847549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685898b-b09f-4800-9c94-ebb66a906339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:14.853591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-4685898b-b09f-4800-9c94-ebb66a906339 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:14.853591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:14.853591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:14.853591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:14.853591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:14.853591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:14.853591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:14.853591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:14.853591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:14.853591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:14.853591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:14.853591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:14.853591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:14.853591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:14.853591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:14.853591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:14.854450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:14.854450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:14.854450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:14.854450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:14.854450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:14.854450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:14.854450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:14.854450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:14.854450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:14.854450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:14.854450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:14.854450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:14.854450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:14.854450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:14.854450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:14.854450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:14.855241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:14.855241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:14.855241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:14.855241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:14.855241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:14.855241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:14.855241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:14.855241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:14.855241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:14.855241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:14.855241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:14.855241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:14.862630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 105/221] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:14.864356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fc3bf-e6db-4bc4-8f71-a87118ea7237 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:14.864890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fc3bf-e6db-4bc4-8f71-a87118ea7237 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:14.865455 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033fc3bf-e6db-4bc4-8f71-a87118ea7237 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:14.866994 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 106/222] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:14.868027 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033fc3bf-e6db-4bc4-8f71-a87118ea7237 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:14.879361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8faef02-36c5-47b4-87cb-5a109355024b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:14.879361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faef02-36c5-47b4-87cb-5a109355024b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:14.879361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faef02-36c5-47b4-87cb-5a109355024b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:14.879361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8faef02-36c5-47b4-87cb-5a109355024b None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:14.879778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8faef02-36c5-47b4-87cb-5a109355024b None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:14.880088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8faef02-36c5-47b4-87cb-5a109355024b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:14.881940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 117/223] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:14 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:14.882916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645b274-ebf5-4ff3-9295-8e3c37ea5502 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:14.883202 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645b274-ebf5-4ff3-9295-8e3c37ea5502 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:14.883479 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645b274-ebf5-4ff3-9295-8e3c37ea5502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:14.904230 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2645b274-ebf5-4ff3-9295-8e3c37ea5502 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:14.905867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2645b274-ebf5-4ff3-9295-8e3c37ea5502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:14.935689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:14.987590 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 118/224] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:14.996159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645b274-ebf5-4ff3-9295-8e3c37ea5502 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:15.016714 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033fc3bf-e6db-4bc4-8f71-a87118ea7237 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:15.055967 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:15.083656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0670d3-8f81-4800-867e-ccf1949d5d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:15.084263 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0670d3-8f81-4800-867e-ccf1949d5d85 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:15.084540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0670d3-8f81-4800-867e-ccf1949d5d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:15.084810 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0670d3-8f81-4800-867e-ccf1949d5d85 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:15.085478 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0670d3-8f81-4800-867e-ccf1949d5d85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:15.085778 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0670d3-8f81-4800-867e-ccf1949d5d85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:15.093734 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0670d3-8f81-4800-867e-ccf1949d5d85 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:15.108577 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 107/225] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:15.116807 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0670d3-8f81-4800-867e-ccf1949d5d85 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:15.185471 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645b274-ebf5-4ff3-9295-8e3c37ea5502 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:15.248809 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360adc7-bc88-42a7-a3b9-6c9c50e3b79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:15.249420 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360adc7-bc88-42a7-a3b9-6c9c50e3b79f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:15.249697 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360adc7-bc88-42a7-a3b9-6c9c50e3b79f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:15.249972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360adc7-bc88-42a7-a3b9-6c9c50e3b79f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:15.250707 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360adc7-bc88-42a7-a3b9-6c9c50e3b79f None admin] RBAC: Adding query filter params (role.id=156b5bb1275a4dbeb61dbdade2c20d21, scope.project.id=d9b6194f22b34cd695de9f04dace7e99, user.id=7f4b926506fe49049f9313f9cd6708cf) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:15.251010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360adc7-bc88-42a7-a3b9-6c9c50e3b79f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:15.252481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360adc7-bc88-42a7-a3b9-6c9c50e3b79f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:15.258903 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 119/226] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:14 2022] GET /identity/v3/role_assignments?user.id=7f4b926506fe49049f9313f9cd6708cf&scope.project.id=d9b6194f22b34cd695de9f04dace7e99&role.id=156b5bb1275a4dbeb61dbdade2c20d21 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:15.262777 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0670d3-8f81-4800-867e-ccf1949d5d85 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:15.304634 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0fd35-7100-45bf-8dbf-c28a91eeefff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:15.305309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0fd35-7100-45bf-8dbf-c28a91eeefff None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:15.305599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0fd35-7100-45bf-8dbf-c28a91eeefff None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:15.305885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0fd35-7100-45bf-8dbf-c28a91eeefff None admin] PATH_INFO: `/v3/users/d39bf2c0c3d24386aa925ba8ecbf4f2c` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:15.307489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0fd35-7100-45bf-8dbf-c28a91eeefff None admin] RBAC: Authorizing `identity:get_user(user_id=d39bf2c0c3d24386aa925ba8ecbf4f2c)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:15.309721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0fd35-7100-45bf-8dbf-c28a91eeefff None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:15.311548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 108/227] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:15 2022] GET /identity/v3/users/d39bf2c0c3d24386aa925ba8ecbf4f2c => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:15.317221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360adc7-bc88-42a7-a3b9-6c9c50e3b79f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:15.336309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0fd35-7100-45bf-8dbf-c28a91eeefff None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:15.359840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:15.387019 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:15.459413 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0fd35-7100-45bf-8dbf-c28a91eeefff None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:15.498562 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b30603-9501-4f1d-b157-1878d58c20d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:15.499207 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b30603-9501-4f1d-b157-1878d58c20d9 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:15.499492 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b30603-9501-4f1d-b157-1878d58c20d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:15.499797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b30603-9501-4f1d-b157-1878d58c20d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:15.511376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b30603-9501-4f1d-b157-1878d58c20d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:15.517119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-f0b30603-9501-4f1d-b157-1878d58c20d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:15.517119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:15.517119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:15.517119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:15.517119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:15.517119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:15.517119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:15.517119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:15.517119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:15.517119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:15.517119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:15.517119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:15.517119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:15.517119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:15.517119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:15.517119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:15.518004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:15.518004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:15.518004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:15.518004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:15.518004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:15.518004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:15.518004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:15.518004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:15.518004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:15.518004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:15.518004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:15.518004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:15.518004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:15.518004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:15.518004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:15.518004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:15.518818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:15.518818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:15.518818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:15.518818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:15.518818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:15.518818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:15.518818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:15.518818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:15.518818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:15.518818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:15.518818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:15.518818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:15.519425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360adc7-bc88-42a7-a3b9-6c9c50e3b79f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:15.519505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 109/228] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:15.524056 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b30603-9501-4f1d-b157-1878d58c20d9 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:15.556944 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10b52f-c6c4-419e-86a5-d82475c2d98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:15.557506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10b52f-c6c4-419e-86a5-d82475c2d98b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:15.557789 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10b52f-c6c4-419e-86a5-d82475c2d98b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:15.558185 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10b52f-c6c4-419e-86a5-d82475c2d98b None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:15.559517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10b52f-c6c4-419e-86a5-d82475c2d98b None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:15.561062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10b52f-c6c4-419e-86a5-d82475c2d98b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:15.563323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 120/229] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:15 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:15.579127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10b52f-c6c4-419e-86a5-d82475c2d98b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:15.636526 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b30603-9501-4f1d-b157-1878d58c20d9 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:15.639017 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:15.694181 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c184f4-0972-4193-b6c1-b4bcae980336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:15.695111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c184f4-0972-4193-b6c1-b4bcae980336 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:15.695399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c184f4-0972-4193-b6c1-b4bcae980336 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:15.695777 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c184f4-0972-4193-b6c1-b4bcae980336 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:15.696717 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c184f4-0972-4193-b6c1-b4bcae980336 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:15.697421 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c184f4-0972-4193-b6c1-b4bcae980336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:15.698936 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c184f4-0972-4193-b6c1-b4bcae980336 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:15.717552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 110/230] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:15.719980 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:15.725694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c184f4-0972-4193-b6c1-b4bcae980336 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:15.733002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10b52f-c6c4-419e-86a5-d82475c2d98b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:15.803237 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f9941-4275-4497-8f87-eef2c4d115d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:15.803978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f9941-4275-4497-8f87-eef2c4d115d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:15.804326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f9941-4275-4497-8f87-eef2c4d115d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:15.804608 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f9941-4275-4497-8f87-eef2c4d115d1 None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99/users/d39bf2c0c3d24386aa925ba8ecbf4f2c/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:15.806595 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f9941-4275-4497-8f87-eef2c4d115d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b6194f22b34cd695de9f04dace7e99, user_id=d39bf2c0c3d24386aa925ba8ecbf4f2c, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:15.808175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f9941-4275-4497-8f87-eef2c4d115d1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:15.848488 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 121/231] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:15 2022] PUT /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99/users/d39bf2c0c3d24386aa925ba8ecbf4f2c/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:19:15.913794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c184f4-0972-4193-b6c1-b4bcae980336 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:15.965256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4509f-8b7c-4e2f-a92d-42c42f0d88a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:15.966040 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4509f-8b7c-4e2f-a92d-42c42f0d88a7 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:15.966324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4509f-8b7c-4e2f-a92d-42c42f0d88a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:15.966602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4509f-8b7c-4e2f-a92d-42c42f0d88a7 None admin] PATH_INFO: `/v3/users/d39bf2c0c3d24386aa925ba8ecbf4f2c` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:15.968355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4509f-8b7c-4e2f-a92d-42c42f0d88a7 None admin] RBAC: Authorizing `identity:get_user(user_id=d39bf2c0c3d24386aa925ba8ecbf4f2c)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:15.969918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4509f-8b7c-4e2f-a92d-42c42f0d88a7 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:15.971545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 111/232] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:15 2022] GET /identity/v3/users/d39bf2c0c3d24386aa925ba8ecbf4f2c => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:15.977118 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f9941-4275-4497-8f87-eef2c4d115d1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:16.110113 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f9941-4275-4497-8f87-eef2c4d115d1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:16.146793 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a2973-324b-41b6-b825-62cf86566478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:16.147440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a2973-324b-41b6-b825-62cf86566478 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:16.147760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a2973-324b-41b6-b825-62cf86566478 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:16.148040 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a2973-324b-41b6-b825-62cf86566478 None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:16.149179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a2973-324b-41b6-b825-62cf86566478 None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:16.150701 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a2973-324b-41b6-b825-62cf86566478 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:16.153124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 122/233] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:15 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:16.158903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4509f-8b7c-4e2f-a92d-42c42f0d88a7 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:16.201775 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:16.279741 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4509f-8b7c-4e2f-a92d-42c42f0d88a7 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:16.321274 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c3485-df90-485a-883b-a932ac8c96f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:16.322374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c3485-df90-485a-883b-a932ac8c96f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:16.322855 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c3485-df90-485a-883b-a932ac8c96f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:16.323381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c3485-df90-485a-883b-a932ac8c96f2 None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99/users/d39bf2c0c3d24386aa925ba8ecbf4f2c/roles/4e002a22056744acb8c40a3236cab404` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:16.329414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c3485-df90-485a-883b-a932ac8c96f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b6194f22b34cd695de9f04dace7e99, user_id=d39bf2c0c3d24386aa925ba8ecbf4f2c, role_id=4e002a22056744acb8c40a3236cab404)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:16.331204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c3485-df90-485a-883b-a932ac8c96f2 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:16.355086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 112/234] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:16 2022] PUT /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99/users/d39bf2c0c3d24386aa925ba8ecbf4f2c/roles/4e002a22056744acb8c40a3236cab404 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:16.480767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5ad6d-3ee4-4498-8b87-bf3c368e4db5 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:16.480956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5ad6d-3ee4-4498-8b87-bf3c368e4db5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:16.481303 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5ad6d-3ee4-4498-8b87-bf3c368e4db5 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:16.482717 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 123/235] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:16.488484 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40dadc-a1d7-4b8c-abb6-05a06fc016a5 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:16.489049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40dadc-a1d7-4b8c-abb6-05a06fc016a5 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:16.489602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40dadc-a1d7-4b8c-abb6-05a06fc016a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:16.516010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-2d40dadc-a1d7-4b8c-abb6-05a06fc016a5 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:16.517592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2d40dadc-a1d7-4b8c-abb6-05a06fc016a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:16.615462 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 113/236] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:16.664580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ab02c-42e4-47bb-b3c4-e5081eb326aa None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:16.664891 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ab02c-42e4-47bb-b3c4-e5081eb326aa None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:16.665148 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ab02c-42e4-47bb-b3c4-e5081eb326aa None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:16.667133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 124/237] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:16.673679 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80e377-cced-4b19-b057-cfd8cdd22779 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:16.674198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80e377-cced-4b19-b057-cfd8cdd22779 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:16.674198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80e377-cced-4b19-b057-cfd8cdd22779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:16.695283 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-2d80e377-cced-4b19-b057-cfd8cdd22779 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:16.696538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2d80e377-cced-4b19-b057-cfd8cdd22779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:16.717627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:16.776069 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 114/238] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:16.944330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ab02c-42e4-47bb-b3c4-e5081eb326aa None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:16.978259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:17.076736 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ab02c-42e4-47bb-b3c4-e5081eb326aa None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:17.111339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53aff9-3c6b-4a27-8472-1f52fcc21c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:17.112013 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53aff9-3c6b-4a27-8472-1f52fcc21c03 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.112295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53aff9-3c6b-4a27-8472-1f52fcc21c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.112571 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53aff9-3c6b-4a27-8472-1f52fcc21c03 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.123558 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53aff9-3c6b-4a27-8472-1f52fcc21c03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:17.128529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-8d53aff9-3c6b-4a27-8472-1f52fcc21c03 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:17.128529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:17.128529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:17.128529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:17.128529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:17.128529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:17.128529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:17.128529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:17.128529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:17.128529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:17.128529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:17.128529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:17.128529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:17.128529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:17.128529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:17.128529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:17.129406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:17.129406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:17.129406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:17.129406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:17.129406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:17.129406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:17.129406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:17.129406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:17.129406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:17.129406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:17.129406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:17.129406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:17.129406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:17.129406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:17.129406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:17.129406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:17.130225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:17.130225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:17.130225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:17.130225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:17.130225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:17.130225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:17.130225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:17.130225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:17.130225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:17.130225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:17.130225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:17.130225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:17.130225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 125/239] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:17.137055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80e377-cced-4b19-b057-cfd8cdd22779 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:17.157215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53aff9-3c6b-4a27-8472-1f52fcc21c03 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:17.270431 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:17.291782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53aff9-3c6b-4a27-8472-1f52fcc21c03 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:17.327319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80e377-cced-4b19-b057-cfd8cdd22779 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:17.366664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49aa47-dc3a-4aee-888d-3217f11577de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:17.367421 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49aa47-dc3a-4aee-888d-3217f11577de None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.367732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49aa47-dc3a-4aee-888d-3217f11577de None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.368005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49aa47-dc3a-4aee-888d-3217f11577de None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.368674 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49aa47-dc3a-4aee-888d-3217f11577de None admin] RBAC: Adding query filter params (name=reader) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:17.368976 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49aa47-dc3a-4aee-888d-3217f11577de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:17.370355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49aa47-dc3a-4aee-888d-3217f11577de None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:17.379730 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 115/240] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:17.384876 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49aa47-dc3a-4aee-888d-3217f11577de None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:17.385861 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1e449-0762-4acc-92c5-72fca310bf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:17.387251 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1e449-0762-4acc-92c5-72fca310bf61 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.387589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1e449-0762-4acc-92c5-72fca310bf61 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.387987 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1e449-0762-4acc-92c5-72fca310bf61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.391442 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1e449-0762-4acc-92c5-72fca310bf61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:17.393842 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1e449-0762-4acc-92c5-72fca310bf61 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:17.396911 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 126/241] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:17.406821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1e449-0762-4acc-92c5-72fca310bf61 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:17.417123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:17.525635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49aa47-dc3a-4aee-888d-3217f11577de None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:17.564599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1e449-0762-4acc-92c5-72fca310bf61 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:17.591502 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27204a-cf31-47e7-826a-96aad68ddf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:17.592098 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27204a-cf31-47e7-826a-96aad68ddf03 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.592378 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27204a-cf31-47e7-826a-96aad68ddf03 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.592652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27204a-cf31-47e7-826a-96aad68ddf03 None admin] PATH_INFO: `/v3/users/e0d9576e407d4cdda0d095853aed28e2` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.606257 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b83c37-3b1b-43ab-ad04-17656b4f0c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:17.606872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b83c37-3b1b-43ab-ad04-17656b4f0c70 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.607153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b83c37-3b1b-43ab-ad04-17656b4f0c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.607428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b83c37-3b1b-43ab-ad04-17656b4f0c70 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.608111 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b83c37-3b1b-43ab-ad04-17656b4f0c70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:17.609839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b83c37-3b1b-43ab-ad04-17656b4f0c70 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:17.621816 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27204a-cf31-47e7-826a-96aad68ddf03 None admin] RBAC: Authorizing `identity:get_user(user_id=e0d9576e407d4cdda0d095853aed28e2)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:17.623299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27204a-cf31-47e7-826a-96aad68ddf03 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:17.625068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 116/242] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:17 2022] GET /identity/v3/users/e0d9576e407d4cdda0d095853aed28e2 => generated 297 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:17.635152 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27204a-cf31-47e7-826a-96aad68ddf03 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:17.654558 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 127/243] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:17 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:19:17.725459 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:17.754406 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27204a-cf31-47e7-826a-96aad68ddf03 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:17.793291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8111b8c-4760-4312-822b-0e602a5b0c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:17.794011 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8111b8c-4760-4312-822b-0e602a5b0c17 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.794291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8111b8c-4760-4312-822b-0e602a5b0c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.794565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8111b8c-4760-4312-822b-0e602a5b0c17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.874970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8111b8c-4760-4312-822b-0e602a5b0c17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:17.876424 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8111b8c-4760-4312-822b-0e602a5b0c17 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:17.920262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3cf8a-e1eb-496c-aac1-7df1496be683 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.920580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3cf8a-e1eb-496c-aac1-7df1496be683 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.920823 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3cf8a-e1eb-496c-aac1-7df1496be683 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.923216 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 128/244] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:17.931027 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3a087-ad78-4429-b1a1-b48b9e5649a0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:17.931317 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3a087-ad78-4429-b1a1-b48b9e5649a0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:17.931596 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3a087-ad78-4429-b1a1-b48b9e5649a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:17.945652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:17.954238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-cac3a087-ad78-4429-b1a1-b48b9e5649a0 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:17.955554 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cac3a087-ad78-4429-b1a1-b48b9e5649a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:17.970648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:18.008495 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 117/245] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:18.014634 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8111b8c-4760-4312-822b-0e602a5b0c17 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:18.064076 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 129/246] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:18.139700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8111b8c-4760-4312-822b-0e602a5b0c17 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:18.184004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db8ff9-3b9f-412e-9ad8-be6f2843f29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:18.184656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db8ff9-3b9f-412e-9ad8-be6f2843f29f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.184939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db8ff9-3b9f-412e-9ad8-be6f2843f29f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.185214 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db8ff9-3b9f-412e-9ad8-be6f2843f29f None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.186467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db8ff9-3b9f-412e-9ad8-be6f2843f29f None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:18.187959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db8ff9-3b9f-412e-9ad8-be6f2843f29f None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:18.194624 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 118/247] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:18 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:18.195992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3a087-ad78-4429-b1a1-b48b9e5649a0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:18.321146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3a087-ad78-4429-b1a1-b48b9e5649a0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:18.362213 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22ab346-1c6e-44e3-be98-dc829bb8aa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:18.362849 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ab346-1c6e-44e3-be98-dc829bb8aa9a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.363113 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ab346-1c6e-44e3-be98-dc829bb8aa9a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.363456 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22ab346-1c6e-44e3-be98-dc829bb8aa9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.364230 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ab346-1c6e-44e3-be98-dc829bb8aa9a None admin] RBAC: Adding query filter params (role.id=0f1253d845894622b8091c95dfd079d3, scope.project.id=d9b6194f22b34cd695de9f04dace7e99, user.id=e0d9576e407d4cdda0d095853aed28e2) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:18.364540 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ab346-1c6e-44e3-be98-dc829bb8aa9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:18.366003 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22ab346-1c6e-44e3-be98-dc829bb8aa9a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:18.369244 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:18.374199 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 130/248] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:18 2022] GET /identity/v3/role_assignments?user.id=e0d9576e407d4cdda0d095853aed28e2&scope.project.id=d9b6194f22b34cd695de9f04dace7e99&role.id=0f1253d845894622b8091c95dfd079d3 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:18.405703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad021d9a-7028-464d-883b-4efd574303cc None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.406295 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad021d9a-7028-464d-883b-4efd574303cc None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.406767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad021d9a-7028-464d-883b-4efd574303cc None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.408867 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 119/249] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:18.414295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9553756-6f24-4138-a4bd-ae0b53acd148 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.414577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9553756-6f24-4138-a4bd-ae0b53acd148 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.414861 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9553756-6f24-4138-a4bd-ae0b53acd148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.427178 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad021d9a-7028-464d-883b-4efd574303cc None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:18.438699 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e9553756-6f24-4138-a4bd-ae0b53acd148 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:18.440645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e9553756-6f24-4138-a4bd-ae0b53acd148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:18.503255 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:18.547123 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 131/250] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:18.602412 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad021d9a-7028-464d-883b-4efd574303cc None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:18.654713 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8381b-aa04-416b-90d1-c55c16013e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:18.655456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8381b-aa04-416b-90d1-c55c16013e45 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.655829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8381b-aa04-416b-90d1-c55c16013e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.656051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8381b-aa04-416b-90d1-c55c16013e45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.676498 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb8381b-aa04-416b-90d1-c55c16013e45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:18.681309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-ccb8381b-aa04-416b-90d1-c55c16013e45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:18.681309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:18.681309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:18.681309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:18.681309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:18.681309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:18.681309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:18.681309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:18.681309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:18.681309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:18.681309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:18.681309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:18.681309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:18.681309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:18.681309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:18.681309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:18.682183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:18.682183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:18.682183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:18.682183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:18.682183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:18.682183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:18.682183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:18.682183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:18.682183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:18.682183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:18.682183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:18.682183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:18.682183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:18.682183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:18.682183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:18.682183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:18.683636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:18.683636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:18.683636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:18.683636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:18.683636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:18.683636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:18.683636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:18.683636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:18.683636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:18.683636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:18.683636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:18.683636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:18.683636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 120/251] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:18.690480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9553756-6f24-4138-a4bd-ae0b53acd148 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:18.841100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9553756-6f24-4138-a4bd-ae0b53acd148 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:18.842605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90f7db-11b3-43de-89d6-fa33aa217905 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.843254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90f7db-11b3-43de-89d6-fa33aa217905 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.843254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90f7db-11b3-43de-89d6-fa33aa217905 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.844767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 121/252] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:18.857941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c875e-6168-4b3b-a4f6-2ff34334a472 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.858271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c875e-6168-4b3b-a4f6-2ff34334a472 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.858500 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c875e-6168-4b3b-a4f6-2ff34334a472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.887974 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:18.897399 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37da18b-24f4-42e4-8147-a17371dab7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:18.898128 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37da18b-24f4-42e4-8147-a17371dab7ce None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:18.898502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37da18b-24f4-42e4-8147-a17371dab7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:18.898870 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37da18b-24f4-42e4-8147-a17371dab7ce None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:18.899651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37da18b-24f4-42e4-8147-a17371dab7ce None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:18.900058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37da18b-24f4-42e4-8147-a17371dab7ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:18.901539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37da18b-24f4-42e4-8147-a17371dab7ce None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:18.911372 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-0d4c875e-6168-4b3b-a4f6-2ff34334a472 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:18.913432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-0d4c875e-6168-4b3b-a4f6-2ff34334a472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:18.924747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 132/253] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:18.925730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37da18b-24f4-42e4-8147-a17371dab7ce None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.000708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 122/254] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:19.002043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c875e-6168-4b3b-a4f6-2ff34334a472 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.048128 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:19.120200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c875e-6168-4b3b-a4f6-2ff34334a472 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:19.157906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c20b14-5390-49bf-9574-225e6dd75852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:19.158918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c20b14-5390-49bf-9574-225e6dd75852 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:19.159403 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c20b14-5390-49bf-9574-225e6dd75852 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:19.159909 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c20b14-5390-49bf-9574-225e6dd75852 None admin] PATH_INFO: `/v3/users/7f4b926506fe49049f9313f9cd6708cf` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:19.161274 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c20b14-5390-49bf-9574-225e6dd75852 None admin] RBAC: Authorizing `identity:get_user(user_id=7f4b926506fe49049f9313f9cd6708cf)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:19.162974 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c20b14-5390-49bf-9574-225e6dd75852 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:19.165025 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 123/255] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:19 2022] GET /identity/v3/users/7f4b926506fe49049f9313f9cd6708cf => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:19.172703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c20b14-5390-49bf-9574-225e6dd75852 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.214888 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37da18b-24f4-42e4-8147-a17371dab7ce None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:19.306861 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd16c2b-2ea0-4bb6-b3a4-e5d4c994b322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:19.307565 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd16c2b-2ea0-4bb6-b3a4-e5d4c994b322 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:19.307822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd16c2b-2ea0-4bb6-b3a4-e5d4c994b322 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:19.308148 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd16c2b-2ea0-4bb6-b3a4-e5d4c994b322 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:19.327055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:19.332828 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd16c2b-2ea0-4bb6-b3a4-e5d4c994b322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:19.335044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c20b14-5390-49bf-9574-225e6dd75852 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:19.338371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-ddd16c2b-2ea0-4bb6-b3a4-e5d4c994b322 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:19.338371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:19.338371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:19.338371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:19.338371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:19.338371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:19.338371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:19.338371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:19.338371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:19.338371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:19.338371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:19.338371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:19.338371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:19.338371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:19.338371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:19.338371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:19.339263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:19.339263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:19.339263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:19.339263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:19.339263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:19.339263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:19.339263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:19.339263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:19.339263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:19.339263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:19.339263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:19.339263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:19.339263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:19.339263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:19.339263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:19.339263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:19.341038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:19.341038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:19.341038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:19.341038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:19.341038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:19.341038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:19.341038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:19.341038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:19.341038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:19.341038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:19.341038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:19.341038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:19.348369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 133/256] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:19.353603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd16c2b-2ea0-4bb6-b3a4-e5d4c994b322 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.390357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07e2e6-9088-427a-ad91-68e4cca21745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:19.391339 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07e2e6-9088-427a-ad91-68e4cca21745 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:19.391907 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07e2e6-9088-427a-ad91-68e4cca21745 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:19.392661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07e2e6-9088-427a-ad91-68e4cca21745 None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:19.394014 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07e2e6-9088-427a-ad91-68e4cca21745 None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:19.395683 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07e2e6-9088-427a-ad91-68e4cca21745 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:19.397543 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 124/257] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:19 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:19.398521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07e2e6-9088-427a-ad91-68e4cca21745 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.441636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:19.493076 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd16c2b-2ea0-4bb6-b3a4-e5d4c994b322 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:19.538615 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545004f-17d3-4c1d-a87e-991be84b80e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:19.539246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545004f-17d3-4c1d-a87e-991be84b80e0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:19.539528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545004f-17d3-4c1d-a87e-991be84b80e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:19.539827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545004f-17d3-4c1d-a87e-991be84b80e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:19.540521 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545004f-17d3-4c1d-a87e-991be84b80e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:19.540791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545004f-17d3-4c1d-a87e-991be84b80e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:19.542172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545004f-17d3-4c1d-a87e-991be84b80e0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:19.551374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07e2e6-9088-427a-ad91-68e4cca21745 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:19.561729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 134/258] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:19.562552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545004f-17d3-4c1d-a87e-991be84b80e0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.598984 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5b317-c38f-4b4d-859e-871f8376bd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:19.599995 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5b317-c38f-4b4d-859e-871f8376bd78 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:19.600478 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5b317-c38f-4b4d-859e-871f8376bd78 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:19.600949 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d5b317-c38f-4b4d-859e-871f8376bd78 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:19.618830 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d5b317-c38f-4b4d-859e-871f8376bd78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:19.624247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-e6d5b317-c38f-4b4d-859e-871f8376bd78 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:19.624247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:19.624247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:19.624247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:19.624247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:19.624247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:19.624247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:19.624247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:19.624247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:19.624247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:19.624247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:19.624247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:19.624247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:19.624247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:19.624247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:19.624247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:19.625119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:19.625119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:19.625119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:19.625119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:19.625119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:19.625119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:19.625119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:19.625119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:19.625119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:19.625119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:19.625119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:19.625119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:19.625119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:19.625119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:19.625119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:19.625119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:19.625917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:19.625917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:19.625917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:19.625917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:19.625917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:19.625917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:19.625917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:19.625917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:19.625917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:19.625917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:19.625917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:19.625917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:19.633506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 125/259] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:19.634515 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5b317-c38f-4b4d-859e-871f8376bd78 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.680559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545004f-17d3-4c1d-a87e-991be84b80e0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:19.695288 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:19.696086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:19.731264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673dbc0-c642-4c55-987f-d96029fda80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:19.732012 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673dbc0-c642-4c55-987f-d96029fda80b None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:19.732298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673dbc0-c642-4c55-987f-d96029fda80b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:19.732580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673dbc0-c642-4c55-987f-d96029fda80b None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99/users/7f4b926506fe49049f9313f9cd6708cf/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:19.734528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673dbc0-c642-4c55-987f-d96029fda80b None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b6194f22b34cd695de9f04dace7e99, user_id=7f4b926506fe49049f9313f9cd6708cf, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:19.736006 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673dbc0-c642-4c55-987f-d96029fda80b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:19.769511 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 135/260] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:19 2022] PUT /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99/users/7f4b926506fe49049f9313f9cd6708cf/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:19.777564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673dbc0-c642-4c55-987f-d96029fda80b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:19.835353 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d5b317-c38f-4b4d-859e-871f8376bd78 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:19.899390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc433ef-a171-4e6d-9356-1c5f3dddb8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:19.900255 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc433ef-a171-4e6d-9356-1c5f3dddb8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:19.900634 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc433ef-a171-4e6d-9356-1c5f3dddb8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:19.901005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc433ef-a171-4e6d-9356-1c5f3dddb8a5 None admin] PATH_INFO: `/v3/users/d39bf2c0c3d24386aa925ba8ecbf4f2c` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:19.903149 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc433ef-a171-4e6d-9356-1c5f3dddb8a5 None admin] RBAC: Authorizing `identity:get_user(user_id=d39bf2c0c3d24386aa925ba8ecbf4f2c)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:19.904837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc433ef-a171-4e6d-9356-1c5f3dddb8a5 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:19.906545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 126/261] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:19 2022] GET /identity/v3/users/d39bf2c0c3d24386aa925ba8ecbf4f2c => generated 283 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:19.913429 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc433ef-a171-4e6d-9356-1c5f3dddb8a5 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:20.023336 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673dbc0-c642-4c55-987f-d96029fda80b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:20.088335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da8628-880b-4f6d-937a-21afeb8d982a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:20.089384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da8628-880b-4f6d-937a-21afeb8d982a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:20.089942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da8628-880b-4f6d-937a-21afeb8d982a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:20.090532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da8628-880b-4f6d-937a-21afeb8d982a None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:20.091281 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da8628-880b-4f6d-937a-21afeb8d982a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:20.091698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da8628-880b-4f6d-937a-21afeb8d982a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:20.093177 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da8628-880b-4f6d-937a-21afeb8d982a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:20.108086 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 136/262] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:20.113477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da8628-880b-4f6d-937a-21afeb8d982a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:20.115545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:20.124149 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc433ef-a171-4e6d-9356-1c5f3dddb8a5 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:20.167852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbaf19-827e-46a9-acc3-fa24c37fc1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:20.168822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbaf19-827e-46a9-acc3-fa24c37fc1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:20.169729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbaf19-827e-46a9-acc3-fa24c37fc1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:20.170355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbaf19-827e-46a9-acc3-fa24c37fc1b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:20.192372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:20.312733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbaf19-827e-46a9-acc3-fa24c37fc1b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:20.314289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbaf19-827e-46a9-acc3-fa24c37fc1b8 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:20.328783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da8628-880b-4f6d-937a-21afeb8d982a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:20.396690 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda25616-b48e-44ff-8d82-2242c80264a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:20.397338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda25616-b48e-44ff-8d82-2242c80264a5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:20.397619 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda25616-b48e-44ff-8d82-2242c80264a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:20.397890 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda25616-b48e-44ff-8d82-2242c80264a5 None admin] PATH_INFO: `/v3/users/d39bf2c0c3d24386aa925ba8ecbf4f2c` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:20.399254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda25616-b48e-44ff-8d82-2242c80264a5 None admin] RBAC: Authorizing `identity:get_user(user_id=d39bf2c0c3d24386aa925ba8ecbf4f2c)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:20.400752 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda25616-b48e-44ff-8d82-2242c80264a5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:20.402747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 137/263] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:20 2022] GET /identity/v3/users/d39bf2c0c3d24386aa925ba8ecbf4f2c => generated 283 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:20.409124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda25616-b48e-44ff-8d82-2242c80264a5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:20.505088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 127/264] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:20.505762 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:20.510887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbaf19-827e-46a9-acc3-fa24c37fc1b8 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:20.622906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda25616-b48e-44ff-8d82-2242c80264a5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:20.628375 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:20.694301 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3368a3a-84e8-4573-8c70-cce9222dc3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:20.694922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3368a3a-84e8-4573-8c70-cce9222dc3bd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:20.695201 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3368a3a-84e8-4573-8c70-cce9222dc3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:20.695473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3368a3a-84e8-4573-8c70-cce9222dc3bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:20.702312 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbaf19-827e-46a9-acc3-fa24c37fc1b8 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:20.780068 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3368a3a-84e8-4573-8c70-cce9222dc3bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:20.781554 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3368a3a-84e8-4573-8c70-cce9222dc3bd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:20.788868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed4d5f-69e8-4f08-b9e6-3fef91a0b162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:20.789509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed4d5f-69e8-4f08-b9e6-3fef91a0b162 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:20.789791 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed4d5f-69e8-4f08-b9e6-3fef91a0b162 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:20.790053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed4d5f-69e8-4f08-b9e6-3fef91a0b162 None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:20.796336 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed4d5f-69e8-4f08-b9e6-3fef91a0b162 None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:20.797807 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed4d5f-69e8-4f08-b9e6-3fef91a0b162 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:20.799478 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 128/265] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:20 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:20.806624 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed4d5f-69e8-4f08-b9e6-3fef91a0b162 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:20.932457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 138/266] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:20.934527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43996af5-4ced-4b7c-b3e7-08dc10f82d80 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:20.934915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43996af5-4ced-4b7c-b3e7-08dc10f82d80 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:20.935206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43996af5-4ced-4b7c-b3e7-08dc10f82d80 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:20.936562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 139/267] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:20.941158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:20.942349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1ea0a-7674-4d24-9158-e43bd7fe7544 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:20.942638 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1ea0a-7674-4d24-9158-e43bd7fe7544 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:20.942915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1ea0a-7674-4d24-9158-e43bd7fe7544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:20.969383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:20.974762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-bee1ea0a-7674-4d24-9158-e43bd7fe7544 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:20.989130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bee1ea0a-7674-4d24-9158-e43bd7fe7544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:21.021323 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed4d5f-69e8-4f08-b9e6-3fef91a0b162 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:21.072426 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd906572-4e12-43a8-a063-439e3c817e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:21.074156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd906572-4e12-43a8-a063-439e3c817e3f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:21.075324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd906572-4e12-43a8-a063-439e3c817e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:21.075891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd906572-4e12-43a8-a063-439e3c817e3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:21.077413 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd906572-4e12-43a8-a063-439e3c817e3f None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=d9b6194f22b34cd695de9f04dace7e99, user.id=d39bf2c0c3d24386aa925ba8ecbf4f2c) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:21.077926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd906572-4e12-43a8-a063-439e3c817e3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:21.079576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd906572-4e12-43a8-a063-439e3c817e3f None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:21.088746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 129/268] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:20 2022] GET /identity/v3/role_assignments?user.id=d39bf2c0c3d24386aa925ba8ecbf4f2c&scope.project.id=d9b6194f22b34cd695de9f04dace7e99&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:21.089893 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd906572-4e12-43a8-a063-439e3c817e3f None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:21.107329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 140/269] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:21.109024 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1927dfc-da31-47f8-b55f-999f97056417 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:21.109524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1927dfc-da31-47f8-b55f-999f97056417 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:21.110390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1927dfc-da31-47f8-b55f-999f97056417 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:21.112022 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 141/270] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:21.118822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac179c-a597-4d56-8171-0ad0cf816c10 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:21.119336 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac179c-a597-4d56-8171-0ad0cf816c10 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:21.119853 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac179c-a597-4d56-8171-0ad0cf816c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:21.158481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4aac179c-a597-4d56-8171-0ad0cf816c10 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:21.160018 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4aac179c-a597-4d56-8171-0ad0cf816c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:21.239105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 142/271] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:21.240743 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46d5c0-53fb-4880-bb44-ed7908fc9448 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:21.241242 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46d5c0-53fb-4880-bb44-ed7908fc9448 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:21.241696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46d5c0-53fb-4880-bb44-ed7908fc9448 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:21.243139 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 143/272] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:21.253792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e7f90-b382-4df7-b273-7185baf7c4dd None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:21.254316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e7f90-b382-4df7-b273-7185baf7c4dd None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:21.254797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e7f90-b382-4df7-b273-7185baf7c4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:21.273646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-cc9e7f90-b382-4df7-b273-7185baf7c4dd None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:21.274858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cc9e7f90-b382-4df7-b273-7185baf7c4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:21.301980 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd906572-4e12-43a8-a063-439e3c817e3f None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:21.338475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:21.346069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 144/273] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:21.352382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:21.363658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cf8fd-ceca-4b96-8fba-358e9dbedde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:21.364629 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cf8fd-ceca-4b96-8fba-358e9dbedde6 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:21.365107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cf8fd-ceca-4b96-8fba-358e9dbedde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:21.365610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cf8fd-ceca-4b96-8fba-358e9dbedde6 None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:21.367027 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49cf8fd-ceca-4b96-8fba-358e9dbedde6 None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:21.368749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49cf8fd-ceca-4b96-8fba-358e9dbedde6 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:21.376139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 130/274] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:21 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:21.381687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e7f90-b382-4df7-b273-7185baf7c4dd None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:21.519833 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e7f90-b382-4df7-b273-7185baf7c4dd None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:21.537033 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cf8fd-ceca-4b96-8fba-358e9dbedde6 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:21.564736 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd8a78-f99c-4617-8436-c19bb2611100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:21.565417 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd8a78-f99c-4617-8436-c19bb2611100 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:21.565700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd8a78-f99c-4617-8436-c19bb2611100 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:21.565974 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd8a78-f99c-4617-8436-c19bb2611100 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:21.566775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd8a78-f99c-4617-8436-c19bb2611100 None admin] RBAC: Adding query filter params (role.id=4e002a22056744acb8c40a3236cab404, scope.project.id=d9b6194f22b34cd695de9f04dace7e99, user.id=d39bf2c0c3d24386aa925ba8ecbf4f2c) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:21.567086 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd8a78-f99c-4617-8436-c19bb2611100 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:21.568728 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd8a78-f99c-4617-8436-c19bb2611100 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:21.577802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 145/275] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:21 2022] GET /identity/v3/role_assignments?user.id=d39bf2c0c3d24386aa925ba8ecbf4f2c&scope.project.id=d9b6194f22b34cd695de9f04dace7e99&role.id=4e002a22056744acb8c40a3236cab404 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:21.578593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd8a78-f99c-4617-8436-c19bb2611100 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:21.653258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:21.670983 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cf8fd-ceca-4b96-8fba-358e9dbedde6 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:21.706966 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd8a78-f99c-4617-8436-c19bb2611100 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:21.710694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b4f0c-41d8-4103-81c9-97a81a42a7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:21.711709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b4f0c-41d8-4103-81c9-97a81a42a7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:21.712218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b4f0c-41d8-4103-81c9-97a81a42a7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:21.712688 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b4f0c-41d8-4103-81c9-97a81a42a7d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:21.714162 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b4f0c-41d8-4103-81c9-97a81a42a7d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:21.716876 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b4f0c-41d8-4103-81c9-97a81a42a7d7 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:21.719030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 131/276] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:21.724424 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b4f0c-41d8-4103-81c9-97a81a42a7d7 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:21.747203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62363e66-cc71-4a84-8d91-7c01d7a66d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:21.748036 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62363e66-cc71-4a84-8d91-7c01d7a66d9c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:21.748324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62363e66-cc71-4a84-8d91-7c01d7a66d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:21.748596 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62363e66-cc71-4a84-8d91-7c01d7a66d9c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:21.761866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62363e66-cc71-4a84-8d91-7c01d7a66d9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:21.768494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-62363e66-cc71-4a84-8d91-7c01d7a66d9c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:21.768494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:21.768494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:21.768494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:21.768494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:21.768494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:21.768494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:21.768494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:21.768494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:21.768494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:21.768494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:21.768494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:21.768494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:21.768494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:21.768494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:21.768494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:21.769409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:21.769409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:21.769409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:21.769409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:21.769409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:21.769409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:21.769409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:21.769409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:21.769409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:21.769409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:21.769409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:21.769409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:21.769409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:21.769409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:21.769409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:21.769409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:21.770351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:21.770351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:21.770351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:21.770351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:21.770351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:21.770351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:21.770351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:21.770351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:21.770351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:21.770351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:21.770351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:21.770351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:21.770351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 146/277] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:21.775388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62363e66-cc71-4a84-8d91-7c01d7a66d9c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:21.828272 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:21.872760 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b4f0c-41d8-4103-81c9-97a81a42a7d7 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:21.898030 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62363e66-cc71-4a84-8d91-7c01d7a66d9c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:21.919790 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:21.925317 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc8ac7-8d0c-4261-8f48-1f4b69ddb2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:21.925971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc8ac7-8d0c-4261-8f48-1f4b69ddb2ab None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:21.926995 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc8ac7-8d0c-4261-8f48-1f4b69ddb2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:21.927282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc8ac7-8d0c-4261-8f48-1f4b69ddb2ab None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:21.927970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc8ac7-8d0c-4261-8f48-1f4b69ddb2ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:21.930066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc8ac7-8d0c-4261-8f48-1f4b69ddb2ab None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:21.944427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33d15f-3292-49c0-b82d-d322e39385cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:21.945126 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33d15f-3292-49c0-b82d-d322e39385cd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:21.945477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33d15f-3292-49c0-b82d-d322e39385cd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:21.945817 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33d15f-3292-49c0-b82d-d322e39385cd None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:21.947003 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33d15f-3292-49c0-b82d-d322e39385cd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:21.947003 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33d15f-3292-49c0-b82d-d322e39385cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:21.948492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33d15f-3292-49c0-b82d-d322e39385cd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:21.959451 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 147/278] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:21.961429 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 132/279] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:21 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:19:21.965039 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33d15f-3292-49c0-b82d-d322e39385cd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:21.966001 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc8ac7-8d0c-4261-8f48-1f4b69ddb2ab None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:22.098658 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33d15f-3292-49c0-b82d-d322e39385cd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:22.121945 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc8ac7-8d0c-4261-8f48-1f4b69ddb2ab None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:22.139720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91108c07-1d18-4aa4-80e3-bd68ddc97e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:22.140556 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91108c07-1d18-4aa4-80e3-bd68ddc97e8c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.140848 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91108c07-1d18-4aa4-80e3-bd68ddc97e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.141427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91108c07-1d18-4aa4-80e3-bd68ddc97e8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.145347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:22.156962 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91108c07-1d18-4aa4-80e3-bd68ddc97e8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:22.162669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-91108c07-1d18-4aa4-80e3-bd68ddc97e8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:22.162669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:22.162669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:22.162669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:22.162669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:22.162669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:22.162669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:22.162669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:22.162669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:22.162669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:22.162669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:22.162669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:22.162669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:22.162669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:22.162669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:22.162669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:22.163551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:22.163551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:22.163551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:22.163551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:22.163551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:22.163551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:22.163551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:22.163551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:22.163551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:22.163551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:22.163551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:22.163551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:22.163551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:22.163551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:22.163551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:22.163551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:22.164432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:22.164432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:22.164432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:22.164432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:22.164432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:22.164432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:22.164432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:22.164432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:22.164432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:22.164432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:22.164432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:22.164432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:22.167730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 148/280] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:22.169731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91108c07-1d18-4aa4-80e3-bd68ddc97e8c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:22.178874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34b2f8-819b-44d6-a7a1-c733109b07fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:22.178874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34b2f8-819b-44d6-a7a1-c733109b07fa None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.178874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34b2f8-819b-44d6-a7a1-c733109b07fa None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.179647 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34b2f8-819b-44d6-a7a1-c733109b07fa None admin] PATH_INFO: `/v3/users/e0d9576e407d4cdda0d095853aed28e2` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.180743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34b2f8-819b-44d6-a7a1-c733109b07fa None admin] RBAC: Authorizing `identity:get_user(user_id=e0d9576e407d4cdda0d095853aed28e2)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:22.182375 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34b2f8-819b-44d6-a7a1-c733109b07fa None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:22.189840 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 133/281] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:21 2022] GET /identity/v3/users/e0d9576e407d4cdda0d095853aed28e2 => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:22.190464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34b2f8-819b-44d6-a7a1-c733109b07fa None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:22.228171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:22.309588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91108c07-1d18-4aa4-80e3-bd68ddc97e8c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:22.323573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34b2f8-819b-44d6-a7a1-c733109b07fa None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:22.353310 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0af63f-f039-42ba-9ec7-544566555248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:22.354027 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0af63f-f039-42ba-9ec7-544566555248 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.354308 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0af63f-f039-42ba-9ec7-544566555248 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.354581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0af63f-f039-42ba-9ec7-544566555248 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.355406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0af63f-f039-42ba-9ec7-544566555248 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:22.356197 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0af63f-f039-42ba-9ec7-544566555248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:22.357738 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0af63f-f039-42ba-9ec7-544566555248 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:22.367711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 149/282] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:22.375060 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0af63f-f039-42ba-9ec7-544566555248 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:22.376097 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5089f8-5a06-4acc-8b47-6f15826f1dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:22.376792 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5089f8-5a06-4acc-8b47-6f15826f1dac None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.377229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5089f8-5a06-4acc-8b47-6f15826f1dac None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.377332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5089f8-5a06-4acc-8b47-6f15826f1dac None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.380432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5089f8-5a06-4acc-8b47-6f15826f1dac None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:22.382113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5089f8-5a06-4acc-8b47-6f15826f1dac None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:22.384460 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 134/283] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:22 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:22.391118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5089f8-5a06-4acc-8b47-6f15826f1dac None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:22.466179 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:22.523861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5089f8-5a06-4acc-8b47-6f15826f1dac None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:22.529127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0af63f-f039-42ba-9ec7-544566555248 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:22.530978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:22.570623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa31a9-052d-4735-8266-bed804589ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:22.571849 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa31a9-052d-4735-8266-bed804589ac5 None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.572338 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa31a9-052d-4735-8266-bed804589ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.573013 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa31a9-052d-4735-8266-bed804589ac5 None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99/users/e0d9576e407d4cdda0d095853aed28e2/roles/0f1253d845894622b8091c95dfd079d3` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.575470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa31a9-052d-4735-8266-bed804589ac5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b6194f22b34cd695de9f04dace7e99, user_id=e0d9576e407d4cdda0d095853aed28e2, role_id=0f1253d845894622b8091c95dfd079d3)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:22.577826 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa31a9-052d-4735-8266-bed804589ac5 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:22.594944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 135/284] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:19:22 2022] PUT /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99/users/e0d9576e407d4cdda0d095853aed28e2/roles/0f1253d845894622b8091c95dfd079d3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:19:22.599825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e90f0-ab10-4f7b-93d3-250e995d01fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:22.599825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e90f0-ab10-4f7b-93d3-250e995d01fe None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.599825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e90f0-ab10-4f7b-93d3-250e995d01fe None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.599825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e90f0-ab10-4f7b-93d3-250e995d01fe None admin] PATH_INFO: `/v3/users/b938eec06cc74e9493e641e16d43f2c8` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.627773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e90f0-ab10-4f7b-93d3-250e995d01fe None admin] RBAC: Authorizing `identity:get_user(user_id=b938eec06cc74e9493e641e16d43f2c8)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:22.629203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e90f0-ab10-4f7b-93d3-250e995d01fe None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:22.631129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 150/285] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:22 2022] GET /identity/v3/users/b938eec06cc74e9493e641e16d43f2c8 => generated 293 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:22.634829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32517b8-fb84-45bc-9d28-9f633393bdbb None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.635239 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32517b8-fb84-45bc-9d28-9f633393bdbb None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.635565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32517b8-fb84-45bc-9d28-9f633393bdbb None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.637440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 136/286] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:22.641560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e90f0-ab10-4f7b-93d3-250e995d01fe None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:22.649253 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3dfbe-2b02-47c0-90c9-acea67ba4453 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.649634 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3dfbe-2b02-47c0-90c9-acea67ba4453 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.649982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3dfbe-2b02-47c0-90c9-acea67ba4453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.682061 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-ccc3dfbe-2b02-47c0-90c9-acea67ba4453 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:22.683960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-ccc3dfbe-2b02-47c0-90c9-acea67ba4453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:22.765797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e90f0-ab10-4f7b-93d3-250e995d01fe None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:22.807658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 137/287] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:22.807867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2e1c2-b605-4253-8d06-2881efeddcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:22.807867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2e1c2-b605-4253-8d06-2881efeddcd0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:22.807867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2e1c2-b605-4253-8d06-2881efeddcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:22.807867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2e1c2-b605-4253-8d06-2881efeddcd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:22.808812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2e1c2-b605-4253-8d06-2881efeddcd0 None admin] RBAC: Adding query filter params (role.id=156b5bb1275a4dbeb61dbdade2c20d21, user.id=b938eec06cc74e9493e641e16d43f2c8, scope.system=all) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:22.808812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2e1c2-b605-4253-8d06-2881efeddcd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:22.810061 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2e1c2-b605-4253-8d06-2881efeddcd0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:22.815816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 151/288] 10.222.0.58 () {56 vars in 1308 bytes} [Wed Aug 24 12:19:22 2022] GET /identity/v3/role_assignments?user.id=b938eec06cc74e9493e641e16d43f2c8&scope.system=all&role.id=156b5bb1275a4dbeb61dbdade2c20d21 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:23.097021 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3dfbe-2b02-47c0-90c9-acea67ba4453 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:23.145151 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:23.204506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3dfbe-2b02-47c0-90c9-acea67ba4453 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:23.254876 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aeec42-f1e7-472f-9a7a-b84eb79ecf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:23.255894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aeec42-f1e7-472f-9a7a-b84eb79ecf0f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:23.256372 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aeec42-f1e7-472f-9a7a-b84eb79ecf0f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:23.256861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aeec42-f1e7-472f-9a7a-b84eb79ecf0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:23.270987 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aeec42-f1e7-472f-9a7a-b84eb79ecf0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:23.276183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-18aeec42-f1e7-472f-9a7a-b84eb79ecf0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:23.276183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:23.276183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:23.276183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:23.276183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:23.276183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:23.276183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:23.276183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:23.276183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:23.276183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:23.276183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:23.276183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:23.276183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:23.276183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:23.276183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:23.276183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:23.277837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:23.277837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:23.277837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:23.277837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:23.277837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:23.277837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:23.277837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:23.277837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:23.277837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:23.277837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:23.277837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:23.277837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:23.277837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:23.277837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:23.277837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:23.277837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:23.278642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:23.278642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:23.278642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:23.278642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:23.278642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:23.278642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:23.278642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:23.278642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:23.278642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:23.278642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:23.278642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:23.278642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:23.285860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 138/289] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:23.287521 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2e1c2-b605-4253-8d06-2881efeddcd0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:23.342760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:23.441137 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2e1c2-b605-4253-8d06-2881efeddcd0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:23.497503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b4a1e-e8a3-4d84-a1dd-c59de5a214c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:23.498118 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b4a1e-e8a3-4d84-a1dd-c59de5a214c7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:23.498395 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b4a1e-e8a3-4d84-a1dd-c59de5a214c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:23.498663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b4a1e-e8a3-4d84-a1dd-c59de5a214c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:23.499325 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b4a1e-e8a3-4d84-a1dd-c59de5a214c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:23.499658 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b4a1e-e8a3-4d84-a1dd-c59de5a214c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:23.501006 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b4a1e-e8a3-4d84-a1dd-c59de5a214c7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:23.515024 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 152/290] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:23.520743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aeec42-f1e7-472f-9a7a-b84eb79ecf0f None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:23.665794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aeec42-f1e7-472f-9a7a-b84eb79ecf0f None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:23.674113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:23.713464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c03a46-1bae-48cd-972f-e96fed39b32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:23.715096 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c03a46-1bae-48cd-972f-e96fed39b32e None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:23.715705 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c03a46-1bae-48cd-972f-e96fed39b32e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:23.716180 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c03a46-1bae-48cd-972f-e96fed39b32e None admin] PATH_INFO: `/v3/users/7f4b926506fe49049f9313f9cd6708cf` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:23.720314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c03a46-1bae-48cd-972f-e96fed39b32e None admin] RBAC: Authorizing `identity:get_user(user_id=7f4b926506fe49049f9313f9cd6708cf)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:23.722006 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c03a46-1bae-48cd-972f-e96fed39b32e None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:23.723870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 139/291] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:23 2022] GET /identity/v3/users/7f4b926506fe49049f9313f9cd6708cf => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:23.729785 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b4a1e-e8a3-4d84-a1dd-c59de5a214c7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:23.862091 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:23.882789 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b4a1e-e8a3-4d84-a1dd-c59de5a214c7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:23.937533 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827282b8-7f60-49f4-a638-afb3ba4bb720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:23.938141 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827282b8-7f60-49f4-a638-afb3ba4bb720 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:23.944090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827282b8-7f60-49f4-a638-afb3ba4bb720 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:23.944376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827282b8-7f60-49f4-a638-afb3ba4bb720 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:24.031456 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827282b8-7f60-49f4-a638-afb3ba4bb720 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:24.032875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827282b8-7f60-49f4-a638-afb3ba4bb720 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:24.148120 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:24.170171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 153/292] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:24.175933 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c03a46-1bae-48cd-972f-e96fed39b32e None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:24.300874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c03a46-1bae-48cd-972f-e96fed39b32e None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:24.352467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32287c87-22d9-4d88-a3da-6f408304fcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:24.353407 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32287c87-22d9-4d88-a3da-6f408304fcbd None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:24.353892 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32287c87-22d9-4d88-a3da-6f408304fcbd None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:24.354342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32287c87-22d9-4d88-a3da-6f408304fcbd None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:24.357559 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32287c87-22d9-4d88-a3da-6f408304fcbd None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:24.359266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32287c87-22d9-4d88-a3da-6f408304fcbd None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:24.361127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 140/293] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:24 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:24.367119 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827282b8-7f60-49f4-a638-afb3ba4bb720 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:24.435124 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1190544-d6b8-4464-bc32-9af5f07589d6 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:24.435875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1190544-d6b8-4464-bc32-9af5f07589d6 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:24.436340 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1190544-d6b8-4464-bc32-9af5f07589d6 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:24.437871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 141/294] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:24.446828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d2b69-702d-42f2-8335-75e6d7c53a14 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:24.447346 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d2b69-702d-42f2-8335-75e6d7c53a14 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:24.447823 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d2b69-702d-42f2-8335-75e6d7c53a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:24.474480 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-7c6d2b69-702d-42f2-8335-75e6d7c53a14 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:24.476891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7c6d2b69-702d-42f2-8335-75e6d7c53a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:24.533531 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827282b8-7f60-49f4-a638-afb3ba4bb720 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:24.550497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:24.603278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 142/295] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:24.604785 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08f848-175c-4272-8830-e11130674f31 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:24.605105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08f848-175c-4272-8830-e11130674f31 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:24.605374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08f848-175c-4272-8830-e11130674f31 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:24.606730 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 143/296] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:24.608580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84150a-87f6-463c-8525-648f76629adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:24.609184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84150a-87f6-463c-8525-648f76629adb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:24.609459 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84150a-87f6-463c-8525-648f76629adb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:24.609727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e84150a-87f6-463c-8525-648f76629adb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:24.610451 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84150a-87f6-463c-8525-648f76629adb None admin] RBAC: Adding query filter params (role.id=156b5bb1275a4dbeb61dbdade2c20d21, scope.project.id=d9b6194f22b34cd695de9f04dace7e99, user.id=7f4b926506fe49049f9313f9cd6708cf) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:24.610753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84150a-87f6-463c-8525-648f76629adb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:24.612237 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e84150a-87f6-463c-8525-648f76629adb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:24.615036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d65bc4-64fe-4284-ad44-119b5a9322a9 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:24.615311 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d65bc4-64fe-4284-ad44-119b5a9322a9 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:24.615575 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d65bc4-64fe-4284-ad44-119b5a9322a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:24.624969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 154/297] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:24 2022] GET /identity/v3/role_assignments?user.id=7f4b926506fe49049f9313f9cd6708cf&scope.project.id=d9b6194f22b34cd695de9f04dace7e99&role.id=156b5bb1275a4dbeb61dbdade2c20d21 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:24.650759 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-a0d65bc4-64fe-4284-ad44-119b5a9322a9 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:24.652708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a0d65bc4-64fe-4284-ad44-119b5a9322a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:24.733725 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 144/298] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:24.971905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84150a-87f6-463c-8525-648f76629adb None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:25.012393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d65bc4-64fe-4284-ad44-119b5a9322a9 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:25.041636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:25.107479 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e84150a-87f6-463c-8525-648f76629adb None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:25.134404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d65bc4-64fe-4284-ad44-119b5a9322a9 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:25.135981 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:25.147645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c1c67-765a-423b-9438-0a7e9871dee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:25.148571 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c1c67-765a-423b-9438-0a7e9871dee6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.149026 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c1c67-765a-423b-9438-0a7e9871dee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.149469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c1c67-765a-423b-9438-0a7e9871dee6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.161924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c1c67-765a-423b-9438-0a7e9871dee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:25.170262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-bd1c1c67-765a-423b-9438-0a7e9871dee6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:25.170262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:25.170262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:25.170262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:25.170262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:25.170262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:25.170262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:25.170262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:25.170262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:25.170262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:25.170262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:25.170262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:25.170262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:25.170262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:25.170262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:25.170262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:25.171160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:25.171160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:25.171160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:25.171160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:25.171160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:25.171160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:25.171160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:25.171160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:25.171160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:25.171160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:25.171160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:25.171160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:25.171160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:25.171160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:25.171160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:25.171160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:25.171988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:25.171988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:25.171988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:25.171988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:25.171988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:25.171988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:25.171988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:25.171988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:25.171988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:25.171988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:25.171988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:25.171988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:25.176440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 155/299] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:25.176560 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e382760-673d-4300-9c8b-39f4145b502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:25.177361 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e382760-673d-4300-9c8b-39f4145b502e None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.177689 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e382760-673d-4300-9c8b-39f4145b502e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.178028 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e382760-673d-4300-9c8b-39f4145b502e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.179343 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e382760-673d-4300-9c8b-39f4145b502e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:25.183968 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e382760-673d-4300-9c8b-39f4145b502e None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:25.183968 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 145/300] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:25.188669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c1c67-765a-423b-9438-0a7e9871dee6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:25.189860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e382760-673d-4300-9c8b-39f4145b502e None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:25.304023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e382760-673d-4300-9c8b-39f4145b502e None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:25.308400 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c1c67-765a-423b-9438-0a7e9871dee6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:25.319732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:25.342212 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ba66c-b529-4bc5-a75e-fa3d43aa9946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:25.342850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ba66c-b529-4bc5-a75e-fa3d43aa9946 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.343118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ba66c-b529-4bc5-a75e-fa3d43aa9946 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.343393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ba66c-b529-4bc5-a75e-fa3d43aa9946 None admin] PATH_INFO: `/v3/groups` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.344048 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ba66c-b529-4bc5-a75e-fa3d43aa9946 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:25.345496 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ba66c-b529-4bc5-a75e-fa3d43aa9946 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:25.348575 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b87411-9ed3-4d66-b0a6-495c3ddfdac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:25.349179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b87411-9ed3-4d66-b0a6-495c3ddfdac3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.349465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b87411-9ed3-4d66-b0a6-495c3ddfdac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.349722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b87411-9ed3-4d66-b0a6-495c3ddfdac3 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.350377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b87411-9ed3-4d66-b0a6-495c3ddfdac3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:25.350676 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b87411-9ed3-4d66-b0a6-495c3ddfdac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:25.352071 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b87411-9ed3-4d66-b0a6-495c3ddfdac3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:25.362454 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 156/301] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:25.363500 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc29ed-7e91-4ea9-a90a-9bf78bb67708 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.363818 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc29ed-7e91-4ea9-a90a-9bf78bb67708 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.364096 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc29ed-7e91-4ea9-a90a-9bf78bb67708 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.365279 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 157/302] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:25.368423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 146/303] 10.222.0.58 () {60 vars in 1134 bytes} [Wed Aug 24 12:19:25 2022] POST /identity/v3/groups => generated 229 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:25.368757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc29ed-7e91-4ea9-a90a-9bf78bb67708 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:25.377642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49a0f4-37c6-4430-9752-6d2f87b0c016 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.378160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49a0f4-37c6-4430-9752-6d2f87b0c016 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.379379 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49a0f4-37c6-4430-9752-6d2f87b0c016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.408102 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-dd49a0f4-37c6-4430-9752-6d2f87b0c016 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:25.410454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-dd49a0f4-37c6-4430-9752-6d2f87b0c016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:25.442523 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:25.493188 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 147/304] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:25.493385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc29ed-7e91-4ea9-a90a-9bf78bb67708 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:25.494452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559da78-e663-406a-a9b7-bac0a23da720 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.494735 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559da78-e663-406a-a9b7-bac0a23da720 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.495008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559da78-e663-406a-a9b7-bac0a23da720 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.496923 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 148/305] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:25.503682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e209b3-3211-417f-8653-f9f5a1eee8c2 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.503682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e209b3-3211-417f-8653-f9f5a1eee8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.503682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e209b3-3211-417f-8653-f9f5a1eee8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.531166 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-a0e209b3-3211-417f-8653-f9f5a1eee8c2 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:25.532657 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a0e209b3-3211-417f-8653-f9f5a1eee8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:25.535680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee51ff5-ff97-432e-83c5-e5285c6b1942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:25.536858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee51ff5-ff97-432e-83c5-e5285c6b1942 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.537286 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee51ff5-ff97-432e-83c5-e5285c6b1942 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.537728 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee51ff5-ff97-432e-83c5-e5285c6b1942 None admin] PATH_INFO: `/v3/users/3de0b80a34824dd6a90b3b1fc64a0f58` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.554150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee51ff5-ff97-432e-83c5-e5285c6b1942 None admin] RBAC: Authorizing `identity:get_user(user_id=3de0b80a34824dd6a90b3b1fc64a0f58)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:25.555769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee51ff5-ff97-432e-83c5-e5285c6b1942 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:25.559838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 158/306] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:25 2022] GET /identity/v3/users/3de0b80a34824dd6a90b3b1fc64a0f58 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:25.563545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee51ff5-ff97-432e-83c5-e5285c6b1942 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:25.602420 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:25.628215 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 149/307] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:25.681223 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee51ff5-ff97-432e-83c5-e5285c6b1942 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:25.717508 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902aa3a2-2a13-4b49-8b17-29112d041b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:25.718181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902aa3a2-2a13-4b49-8b17-29112d041b38 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.718469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902aa3a2-2a13-4b49-8b17-29112d041b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.718746 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902aa3a2-2a13-4b49-8b17-29112d041b38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.719480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902aa3a2-2a13-4b49-8b17-29112d041b38 None admin] RBAC: Adding query filter params (role.id=0f1253d845894622b8091c95dfd079d3, user.id=3de0b80a34824dd6a90b3b1fc64a0f58, scope.system=all) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:25.719818 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902aa3a2-2a13-4b49-8b17-29112d041b38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:25.721266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902aa3a2-2a13-4b49-8b17-29112d041b38 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:25.726789 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 159/308] 10.222.0.58 () {56 vars in 1308 bytes} [Wed Aug 24 12:19:25 2022] GET /identity/v3/role_assignments?user.id=3de0b80a34824dd6a90b3b1fc64a0f58&scope.system=all&role.id=0f1253d845894622b8091c95dfd079d3 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:25.766640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e209b3-3211-417f-8653-f9f5a1eee8c2 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:25.869943 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:25.883697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e209b3-3211-417f-8653-f9f5a1eee8c2 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:25.920388 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812618e-c181-46be-8dc3-435c87a6b9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:25.921037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812618e-c181-46be-8dc3-435c87a6b9dd None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:25.921318 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812618e-c181-46be-8dc3-435c87a6b9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:25.921594 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812618e-c181-46be-8dc3-435c87a6b9dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:25.927302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902aa3a2-2a13-4b49-8b17-29112d041b38 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:25.934256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3812618e-c181-46be-8dc3-435c87a6b9dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:25.940049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-3812618e-c181-46be-8dc3-435c87a6b9dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:25.940049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:25.940049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:25.940049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:25.940049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:25.940049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:25.940049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:25.940049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:25.940049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:25.940049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:25.940049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:25.940049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:25.940049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:25.940049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:25.940049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:25.940049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:25.941075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:25.941075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:25.941075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:25.941075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:25.941075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:25.941075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:25.941075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:25.941075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:25.941075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:25.941075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:25.941075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:25.941075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:25.941075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:25.941075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:25.941075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:25.941075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:25.941926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:25.941926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:25.941926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:25.941926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:25.941926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:25.941926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:25.941926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:25.941926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:25.941926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:25.941926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:25.941926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:25.941926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:25.941926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 150/309] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:25.947750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812618e-c181-46be-8dc3-435c87a6b9dd None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:26.046959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:26.057136 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902aa3a2-2a13-4b49-8b17-29112d041b38 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:26.076230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812618e-c181-46be-8dc3-435c87a6b9dd None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:26.097563 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece23732-0165-482e-bfe1-337541c549d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:26.098346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece23732-0165-482e-bfe1-337541c549d2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:26.098725 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece23732-0165-482e-bfe1-337541c549d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:26.099097 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece23732-0165-482e-bfe1-337541c549d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:26.110265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece23732-0165-482e-bfe1-337541c549d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:26.115271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-ece23732-0165-482e-bfe1-337541c549d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:26.115271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:26.115271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:26.115271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:26.115271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:26.115271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:26.115271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:26.115271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:26.115271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:26.115271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:26.115271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:26.115271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:26.115271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:26.115271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:26.115271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:26.115271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:26.116839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:26.116839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:26.116839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:26.116839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:26.116839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:26.116839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:26.116839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:26.116839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:26.116839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:26.116839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:26.116839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:26.116839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:26.116839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:26.116839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:26.116839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:26.116839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:26.118678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:26.118678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:26.118678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:26.118678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:26.118678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:26.118678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:26.118678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:26.118678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:26.118678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:26.118678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:26.118678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:26.118678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:26.120165 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28f038-8f9d-4348-a149-c9fbf8365a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:26.120165 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28f038-8f9d-4348-a149-c9fbf8365a48 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:26.120165 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28f038-8f9d-4348-a149-c9fbf8365a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:26.120165 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28f038-8f9d-4348-a149-c9fbf8365a48 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:26.121248 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28f038-8f9d-4348-a149-c9fbf8365a48 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:26.123659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28f038-8f9d-4348-a149-c9fbf8365a48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:26.123659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28f038-8f9d-4348-a149-c9fbf8365a48 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:26.123868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 160/310] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:26.125032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece23732-0165-482e-bfe1-337541c549d2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:26.133410 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 151/311] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:26.139042 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28f038-8f9d-4348-a149-c9fbf8365a48 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:26.160597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:26.239171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece23732-0165-482e-bfe1-337541c549d2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:26.275577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d708f7-ca76-4b75-bbd3-e55308e6654f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:26.276484 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d708f7-ca76-4b75-bbd3-e55308e6654f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:26.277140 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d708f7-ca76-4b75-bbd3-e55308e6654f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:26.277706 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d708f7-ca76-4b75-bbd3-e55308e6654f None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:26.278559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d708f7-ca76-4b75-bbd3-e55308e6654f None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:26.279037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d708f7-ca76-4b75-bbd3-e55308e6654f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:26.280651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d708f7-ca76-4b75-bbd3-e55308e6654f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:26.281659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28f038-8f9d-4348-a149-c9fbf8365a48 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:26.295065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 161/312] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:26.296676 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d708f7-ca76-4b75-bbd3-e55308e6654f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:26.316236 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:26.323922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa8ed5-2654-473f-843d-443247faff9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:26.324557 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa8ed5-2654-473f-843d-443247faff9a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:26.325096 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa8ed5-2654-473f-843d-443247faff9a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:26.325369 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa8ed5-2654-473f-843d-443247faff9a None admin] PATH_INFO: `/v3/users/e0d9576e407d4cdda0d095853aed28e2` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:26.326660 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa8ed5-2654-473f-843d-443247faff9a None admin] RBAC: Authorizing `identity:get_user(user_id=e0d9576e407d4cdda0d095853aed28e2)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:26.328765 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa8ed5-2654-473f-843d-443247faff9a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:26.331271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 152/313] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:26 2022] GET /identity/v3/users/e0d9576e407d4cdda0d095853aed28e2 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:26.336717 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa8ed5-2654-473f-843d-443247faff9a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:26.412089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d708f7-ca76-4b75-bbd3-e55308e6654f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:26.448779 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:26.449034 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f0090-e90f-461a-abce-3e80c5ad96ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:26.450308 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f0090-e90f-461a-abce-3e80c5ad96ef None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:26.450742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f0090-e90f-461a-abce-3e80c5ad96ef None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:26.451205 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f0090-e90f-461a-abce-3e80c5ad96ef None admin] PATH_INFO: `/v3/users/b938eec06cc74e9493e641e16d43f2c8` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:26.452647 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f0090-e90f-461a-abce-3e80c5ad96ef None admin] RBAC: Authorizing `identity:get_user(user_id=b938eec06cc74e9493e641e16d43f2c8)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:26.454235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f0090-e90f-461a-abce-3e80c5ad96ef None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:26.460932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 162/314] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:26 2022] GET /identity/v3/users/b938eec06cc74e9493e641e16d43f2c8 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:26.462358 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f0090-e90f-461a-abce-3e80c5ad96ef None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:26.474285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa8ed5-2654-473f-843d-443247faff9a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:26.525081 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221caf3b-1f88-4149-881d-3639358fcefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:26.526028 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221caf3b-1f88-4149-881d-3639358fcefd None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:26.526319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221caf3b-1f88-4149-881d-3639358fcefd None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:26.526592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221caf3b-1f88-4149-881d-3639358fcefd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:26.556659 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:26.577168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f0090-e90f-461a-abce-3e80c5ad96ef None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:26.615108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4183aa7-2e2b-494e-a689-9d361baec08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:26.616301 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4183aa7-2e2b-494e-a689-9d361baec08a None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:26.617101 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4183aa7-2e2b-494e-a689-9d361baec08a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:26.617802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4183aa7-2e2b-494e-a689-9d361baec08a None admin] PATH_INFO: `/v3/system/users/b938eec06cc74e9493e641e16d43f2c8/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:26.619412 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4183aa7-2e2b-494e-a689-9d361baec08a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b938eec06cc74e9493e641e16d43f2c8, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:26.622214 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4183aa7-2e2b-494e-a689-9d361baec08a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:26.623917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221caf3b-1f88-4149-881d-3639358fcefd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:26.625716 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221caf3b-1f88-4149-881d-3639358fcefd None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:26.640392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 163/315] 10.222.0.58 () {58 vars in 1334 bytes} [Wed Aug 24 12:19:26 2022] PUT /identity/v3/system/users/b938eec06cc74e9493e641e16d43f2c8/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:26.724777 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:26.756097 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 153/316] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:19:26.761840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4183aa7-2e2b-494e-a689-9d361baec08a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:26.871884 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4183aa7-2e2b-494e-a689-9d361baec08a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:26.909016 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722581c-1609-48c7-81bc-f1c1469e05d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:26.909654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722581c-1609-48c7-81bc-f1c1469e05d6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:26.909935 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722581c-1609-48c7-81bc-f1c1469e05d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:26.910211 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722581c-1609-48c7-81bc-f1c1469e05d6 None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:26.911337 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5722581c-1609-48c7-81bc-f1c1469e05d6 None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:26.912827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5722581c-1609-48c7-81bc-f1c1469e05d6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:26.914520 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 164/317] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:26 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:26.924741 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221caf3b-1f88-4149-881d-3639358fcefd None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:27.032687 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221caf3b-1f88-4149-881d-3639358fcefd None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:27.069604 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d5da8-58ac-4fd1-acc8-0a695cacbbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:27.070276 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d5da8-58ac-4fd1-acc8-0a695cacbbeb None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:27.070556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d5da8-58ac-4fd1-acc8-0a695cacbbeb None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:27.070832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d5da8-58ac-4fd1-acc8-0a695cacbbeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:27.071640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d5da8-58ac-4fd1-acc8-0a695cacbbeb None admin] RBAC: Adding query filter params (role.id=0f1253d845894622b8091c95dfd079d3, scope.project.id=d9b6194f22b34cd695de9f04dace7e99, user.id=e0d9576e407d4cdda0d095853aed28e2) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:27.071966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d5da8-58ac-4fd1-acc8-0a695cacbbeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:27.073415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d5da8-58ac-4fd1-acc8-0a695cacbbeb None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:27.080931 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 154/318] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:26 2022] GET /identity/v3/role_assignments?user.id=e0d9576e407d4cdda0d095853aed28e2&scope.project.id=d9b6194f22b34cd695de9f04dace7e99&role.id=0f1253d845894622b8091c95dfd079d3 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:27.471299 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088965a-dae9-4502-915b-caa7c4caf5cb None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:27.471584 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088965a-dae9-4502-915b-caa7c4caf5cb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:27.471908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088965a-dae9-4502-915b-caa7c4caf5cb None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:27.473531 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 165/319] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:27.478665 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866faf64-4dcf-4162-ba14-ad4008d4a169 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:27.479105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866faf64-4dcf-4162-ba14-ad4008d4a169 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:27.479220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866faf64-4dcf-4162-ba14-ad4008d4a169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:27.497652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-866faf64-4dcf-4162-ba14-ad4008d4a169 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:27.498856 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-866faf64-4dcf-4162-ba14-ad4008d4a169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:27.540439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:27.571721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 155/320] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:27.698883 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf72b9-4961-4f46-b543-f32dfa9c6dfb None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:27.699382 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf72b9-4961-4f46-b543-f32dfa9c6dfb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:27.699457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf72b9-4961-4f46-b543-f32dfa9c6dfb None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:27.701217 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 166/321] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:27.706093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb608422-2aa4-41c5-9eeb-58856a9f4257 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:27.706389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb608422-2aa4-41c5-9eeb-58856a9f4257 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:27.706666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb608422-2aa4-41c5-9eeb-58856a9f4257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:27.725371 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-fb608422-2aa4-41c5-9eeb-58856a9f4257 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:27.726491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fb608422-2aa4-41c5-9eeb-58856a9f4257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:27.802244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 156/322] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:27.843859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf72b9-4961-4f46-b543-f32dfa9c6dfb None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:27.862783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:27.951329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf72b9-4961-4f46-b543-f32dfa9c6dfb None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:27.985432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9a199-50db-4e19-adbd-0a0940e8da21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:27.986000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9a199-50db-4e19-adbd-0a0940e8da21 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:27.986293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9a199-50db-4e19-adbd-0a0940e8da21 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:27.986557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9a199-50db-4e19-adbd-0a0940e8da21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:27.987664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9a199-50db-4e19-adbd-0a0940e8da21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:27.989121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9a199-50db-4e19-adbd-0a0940e8da21 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:27.990868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 167/323] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:27.996299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb608422-2aa4-41c5-9eeb-58856a9f4257 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:28.078608 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9a199-50db-4e19-adbd-0a0940e8da21 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:28.106373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb608422-2aa4-41c5-9eeb-58856a9f4257 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:28.109769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:28.144888 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eed215-a635-41eb-bfd9-fc8f86dda0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:28.145497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eed215-a635-41eb-bfd9-fc8f86dda0bb None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:28.145774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eed215-a635-41eb-bfd9-fc8f86dda0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:28.146041 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eed215-a635-41eb-bfd9-fc8f86dda0bb None admin] PATH_INFO: `/v3/groups` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:28.146662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eed215-a635-41eb-bfd9-fc8f86dda0bb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:28.148131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eed215-a635-41eb-bfd9-fc8f86dda0bb None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:28.165207 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 157/324] 10.222.0.58 () {60 vars in 1134 bytes} [Wed Aug 24 12:19:27 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:28.187936 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:28.208255 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9a199-50db-4e19-adbd-0a0940e8da21 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:28.246339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23395a66-5e3d-45f9-be95-12289c1a97e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:28.246906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23395a66-5e3d-45f9-be95-12289c1a97e7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:28.247178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23395a66-5e3d-45f9-be95-12289c1a97e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:28.247444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23395a66-5e3d-45f9-be95-12289c1a97e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:28.259073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23395a66-5e3d-45f9-be95-12289c1a97e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:28.265801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-23395a66-5e3d-45f9-be95-12289c1a97e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:28.265801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:28.265801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:28.265801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:28.265801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:28.265801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:28.265801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:28.265801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:28.265801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:28.265801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:28.265801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:28.265801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:28.265801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:28.265801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:28.265801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:28.265801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:28.266745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:28.266745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:28.266745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:28.266745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:28.266745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:28.266745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:28.266745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:28.266745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:28.266745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:28.266745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:28.266745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:28.266745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:28.266745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:28.266745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:28.266745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:28.266745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:28.267551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:28.267551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:28.267551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:28.267551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:28.267551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:28.267551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:28.267551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:28.267551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:28.267551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:28.267551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:28.267551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:28.267551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:28.271849 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 168/325] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:28.273143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eed215-a635-41eb-bfd9-fc8f86dda0bb None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:28.388129 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eed215-a635-41eb-bfd9-fc8f86dda0bb None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:28.428420 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04f8ce-ec63-4839-a92b-d0340f13db17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:28.433464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04f8ce-ec63-4839-a92b-d0340f13db17 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:28.434183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04f8ce-ec63-4839-a92b-d0340f13db17 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:28.434680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04f8ce-ec63-4839-a92b-d0340f13db17 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:28.435571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04f8ce-ec63-4839-a92b-d0340f13db17 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:28.436187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04f8ce-ec63-4839-a92b-d0340f13db17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:28.437779 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04f8ce-ec63-4839-a92b-d0340f13db17 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:28.448491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 158/326] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:28.455162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23395a66-5e3d-45f9-be95-12289c1a97e7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:28.569138 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23395a66-5e3d-45f9-be95-12289c1a97e7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:28.606001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2000dce-c211-45d5-9fa9-e0a6f5b1ae97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:28.606877 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2000dce-c211-45d5-9fa9-e0a6f5b1ae97 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:28.607157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2000dce-c211-45d5-9fa9-e0a6f5b1ae97 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:28.607428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2000dce-c211-45d5-9fa9-e0a6f5b1ae97 None admin] PATH_INFO: `/v3/users/3de0b80a34824dd6a90b3b1fc64a0f58` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:28.608622 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2000dce-c211-45d5-9fa9-e0a6f5b1ae97 None admin] RBAC: Authorizing `identity:get_user(user_id=3de0b80a34824dd6a90b3b1fc64a0f58)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:28.610142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2000dce-c211-45d5-9fa9-e0a6f5b1ae97 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:28.615692 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 169/327] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:28 2022] GET /identity/v3/users/3de0b80a34824dd6a90b3b1fc64a0f58 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:28.619921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04f8ce-ec63-4839-a92b-d0340f13db17 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:28.643834 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460c4ff-4761-4fcf-a4dd-57c1e8535a84 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:28.644158 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460c4ff-4761-4fcf-a4dd-57c1e8535a84 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:28.644386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460c4ff-4761-4fcf-a4dd-57c1e8535a84 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:28.645748 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 170/328] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:28.650938 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cce2bb-3437-4769-a6e5-53aa689c638b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:28.651212 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cce2bb-3437-4769-a6e5-53aa689c638b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:28.651485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cce2bb-3437-4769-a6e5-53aa689c638b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:28.659613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:28.680795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b3cce2bb-3437-4769-a6e5-53aa689c638b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:28.682369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b3cce2bb-3437-4769-a6e5-53aa689c638b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:28.692876 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:28.768105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04f8ce-ec63-4839-a92b-d0340f13db17 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:28.776389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 171/329] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:28.814098 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f609a90-91f9-42f5-a823-2c62639596da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:28.814814 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f609a90-91f9-42f5-a823-2c62639596da None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:28.815097 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f609a90-91f9-42f5-a823-2c62639596da None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:28.815369 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f609a90-91f9-42f5-a823-2c62639596da None admin] PATH_INFO: `/v3/system/users/3de0b80a34824dd6a90b3b1fc64a0f58/roles/0f1253d845894622b8091c95dfd079d3` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:28.816834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f609a90-91f9-42f5-a823-2c62639596da None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3de0b80a34824dd6a90b3b1fc64a0f58, role_id=0f1253d845894622b8091c95dfd079d3)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:28.818263 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f609a90-91f9-42f5-a823-2c62639596da None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:28.828504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 159/330] 10.222.0.58 () {58 vars in 1334 bytes} [Wed Aug 24 12:19:28 2022] PUT /identity/v3/system/users/3de0b80a34824dd6a90b3b1fc64a0f58/roles/0f1253d845894622b8091c95dfd079d3 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:29.087738 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cce2bb-3437-4769-a6e5-53aa689c638b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:29.209132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cce2bb-3437-4769-a6e5-53aa689c638b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:29.246678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee157f-d324-4ba4-b8f4-c5b555fa8ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:29.247321 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee157f-d324-4ba4-b8f4-c5b555fa8ddf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:29.247604 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee157f-d324-4ba4-b8f4-c5b555fa8ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:29.247901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee157f-d324-4ba4-b8f4-c5b555fa8ddf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:29.258666 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee157f-d324-4ba4-b8f4-c5b555fa8ddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:29.263688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-20ee157f-d324-4ba4-b8f4-c5b555fa8ddf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:29.263688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:29.263688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:29.263688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:29.263688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:29.263688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:29.263688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:29.263688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:29.263688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:29.263688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:29.263688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:29.263688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:29.263688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:29.263688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:29.263688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:29.263688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:29.264560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:29.264560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:29.264560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:29.264560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:29.264560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:29.264560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:29.264560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:29.264560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:29.264560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:29.264560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:29.264560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:29.264560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:29.264560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:29.264560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:29.264560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:29.264560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:29.265800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:29.265800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:29.265800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:29.265800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:29.265800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:29.265800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:29.265800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:29.265800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:29.265800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:29.265800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:29.265800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:29.265800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:29.265800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 172/331] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:29.270925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f609a90-91f9-42f5-a823-2c62639596da None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:29.388066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:29.402971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f609a90-91f9-42f5-a823-2c62639596da None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:29.441659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216895f0-c1c1-4776-a70f-ec91c9d325e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:29.442270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216895f0-c1c1-4776-a70f-ec91c9d325e4 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:29.442553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216895f0-c1c1-4776-a70f-ec91c9d325e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:29.442827 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216895f0-c1c1-4776-a70f-ec91c9d325e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:29.443494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216895f0-c1c1-4776-a70f-ec91c9d325e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:29.443821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216895f0-c1c1-4776-a70f-ec91c9d325e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:29.445201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216895f0-c1c1-4776-a70f-ec91c9d325e4 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:29.457922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 160/332] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:29.463446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee157f-d324-4ba4-b8f4-c5b555fa8ddf None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:29.467463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:29.585417 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee157f-d324-4ba4-b8f4-c5b555fa8ddf None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:29.625743 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8e0ae-d4a5-4d7b-9813-8cd48df51784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:29.626770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8e0ae-d4a5-4d7b-9813-8cd48df51784 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:29.627241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8e0ae-d4a5-4d7b-9813-8cd48df51784 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:29.627726 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8e0ae-d4a5-4d7b-9813-8cd48df51784 None admin] PATH_INFO: `/v3/users/b938eec06cc74e9493e641e16d43f2c8` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:29.629073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8e0ae-d4a5-4d7b-9813-8cd48df51784 None admin] RBAC: Authorizing `identity:get_user(user_id=b938eec06cc74e9493e641e16d43f2c8)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:29.630717 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8e0ae-d4a5-4d7b-9813-8cd48df51784 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:29.632863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 173/333] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:29 2022] GET /identity/v3/users/b938eec06cc74e9493e641e16d43f2c8 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:29.639163 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216895f0-c1c1-4776-a70f-ec91c9d325e4 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:29.769898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216895f0-c1c1-4776-a70f-ec91c9d325e4 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:29.811800 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aad9c3-8c58-4025-9918-457417239b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:29.812430 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aad9c3-8c58-4025-9918-457417239b13 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:29.812716 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aad9c3-8c58-4025-9918-457417239b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:29.812992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aad9c3-8c58-4025-9918-457417239b13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:29.813735 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aad9c3-8c58-4025-9918-457417239b13 None admin] RBAC: Adding query filter params (role.id=156b5bb1275a4dbeb61dbdade2c20d21, user.id=b938eec06cc74e9493e641e16d43f2c8, scope.system=all) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:29.814047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aad9c3-8c58-4025-9918-457417239b13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:29.815498 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aad9c3-8c58-4025-9918-457417239b13 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:29.833880 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 161/334] 10.222.0.58 () {56 vars in 1308 bytes} [Wed Aug 24 12:19:29 2022] GET /identity/v3/role_assignments?user.id=b938eec06cc74e9493e641e16d43f2c8&scope.system=all&role.id=156b5bb1275a4dbeb61dbdade2c20d21 => generated 510 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:30.940886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f9d2a-712f-4ed9-a397-52060995f235 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:30.941827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f9d2a-712f-4ed9-a397-52060995f235 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:30.942296 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f9d2a-712f-4ed9-a397-52060995f235 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:30.944132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 174/335] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:30.951140 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c8229-7404-4a22-8e5f-12bf9185e081 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:30.951641 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c8229-7404-4a22-8e5f-12bf9185e081 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:30.951746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c8229-7404-4a22-8e5f-12bf9185e081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:30.981428 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-e50c8229-7404-4a22-8e5f-12bf9185e081 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:30.982882 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e50c8229-7404-4a22-8e5f-12bf9185e081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:30.987943 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:30.995058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7a41c-335e-4170-a9f1-3124809837d3 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:30.996465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7a41c-335e-4170-a9f1-3124809837d3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:30.997229 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7a41c-335e-4170-a9f1-3124809837d3 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:31.003714 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 175/336] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:31.009936 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067b5e2-34ac-42b2-86f1-cece0806ffd3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:31.010680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067b5e2-34ac-42b2-86f1-cece0806ffd3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:31.011729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067b5e2-34ac-42b2-86f1-cece0806ffd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:31.049291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5067b5e2-34ac-42b2-86f1-cece0806ffd3 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:31.050878 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5067b5e2-34ac-42b2-86f1-cece0806ffd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:31.088148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 162/337] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:31.176865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:31.185083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 176/338] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:31.686248 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c8229-7404-4a22-8e5f-12bf9185e081 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:31.730265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067b5e2-34ac-42b2-86f1-cece0806ffd3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:31.813482 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50c8229-7404-4a22-8e5f-12bf9185e081 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:31.858484 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e45b6-d0d9-4574-9a75-eb68e6650da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:31.859364 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e45b6-d0d9-4574-9a75-eb68e6650da6 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:31.859833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e45b6-d0d9-4574-9a75-eb68e6650da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:31.860244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e45b6-d0d9-4574-9a75-eb68e6650da6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:31.868366 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:31.873466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e45b6-d0d9-4574-9a75-eb68e6650da6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:31.886731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-bb0e45b6-d0d9-4574-9a75-eb68e6650da6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:31.886731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:31.886731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:31.886731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:31.886731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:31.886731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:31.886731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:31.886731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:31.886731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:31.886731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:31.886731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:31.886731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:31.886731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:31.886731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:31.886731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:31.886731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:31.888156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:31.888156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:31.888156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:31.888156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:31.888156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:31.888156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:31.888156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:31.888156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:31.888156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:31.888156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:31.888156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:31.888156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:31.888156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:31.888156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:31.888156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:31.888156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:31.888972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:31.888972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:31.888972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:31.888972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:31.888972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:31.888972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:31.888972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:31.888972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:31.888972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:31.888972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:31.888972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:31.888972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:31.888972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 163/339] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:31.893685 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e45b6-d0d9-4574-9a75-eb68e6650da6 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:31.931411 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067b5e2-34ac-42b2-86f1-cece0806ffd3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:32.001228 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429619cd-a1ce-4cf0-9020-ba8b33028750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:32.001825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429619cd-a1ce-4cf0-9020-ba8b33028750 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:32.002063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429619cd-a1ce-4cf0-9020-ba8b33028750 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:32.002337 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429619cd-a1ce-4cf0-9020-ba8b33028750 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:32.025543 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429619cd-a1ce-4cf0-9020-ba8b33028750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:32.031125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-429619cd-a1ce-4cf0-9020-ba8b33028750 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:32.031125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:32.031125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:32.031125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:32.031125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:32.031125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:32.031125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:32.031125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:32.031125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:32.031125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:32.031125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:32.031125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:32.031125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:32.031125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:32.031125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:32.031125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:32.032065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:32.032065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:32.032065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:32.032065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:32.032065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:32.032065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:32.032065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:32.032065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:32.032065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:32.032065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:32.032065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:32.032065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:32.032065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:32.032065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:32.032065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:32.032065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:32.032863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:32.032863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:32.032863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:32.032863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:32.032863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:32.032863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:32.032863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:32.032863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:32.032863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:32.032863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:32.032863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:32.032863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:32.040910 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 177/340] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:32.059676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e45b6-d0d9-4574-9a75-eb68e6650da6 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:32.061528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429619cd-a1ce-4cf0-9020-ba8b33028750 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:32.099319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f11208-d679-4a7f-a5ed-5d9b5201ba32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:32.099964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f11208-d679-4a7f-a5ed-5d9b5201ba32 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:32.100242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f11208-d679-4a7f-a5ed-5d9b5201ba32 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:32.100513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f11208-d679-4a7f-a5ed-5d9b5201ba32 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:32.101194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f11208-d679-4a7f-a5ed-5d9b5201ba32 None admin] RBAC: Adding query filter params (name=member) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:32.101505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f11208-d679-4a7f-a5ed-5d9b5201ba32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:32.102914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f11208-d679-4a7f-a5ed-5d9b5201ba32 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:32.112635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 164/341] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:32.121486 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f11208-d679-4a7f-a5ed-5d9b5201ba32 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:32.197465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:32.201374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:32.255953 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f11208-d679-4a7f-a5ed-5d9b5201ba32 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:32.256868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429619cd-a1ce-4cf0-9020-ba8b33028750 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:32.292704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c4de3-3641-4f0a-89de-f7177b3e401e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:32.293731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c4de3-3641-4f0a-89de-f7177b3e401e None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:32.294019 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c4de3-3641-4f0a-89de-f7177b3e401e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:32.294312 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c4de3-3641-4f0a-89de-f7177b3e401e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:32.295255 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa02892-7765-4af6-8a40-127680ab340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:32.296047 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa02892-7765-4af6-8a40-127680ab340a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:32.296465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa02892-7765-4af6-8a40-127680ab340a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:32.296875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa02892-7765-4af6-8a40-127680ab340a None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:32.297700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa02892-7765-4af6-8a40-127680ab340a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:32.298141 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa02892-7765-4af6-8a40-127680ab340a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:32.299694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa02892-7765-4af6-8a40-127680ab340a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:32.309471 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 178/342] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:32.310890 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7024c-42c9-49b7-8e06-21bc754320dd None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:32.311345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7024c-42c9-49b7-8e06-21bc754320dd None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:32.311785 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7024c-42c9-49b7-8e06-21bc754320dd None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:32.313150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 179/343] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:32.314303 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6396a26-dd08-490b-ba98-f6726da69403 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:32.314726 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6396a26-dd08-490b-ba98-f6726da69403 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:32.315140 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6396a26-dd08-490b-ba98-f6726da69403 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:32.320643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 180/344] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:32.322282 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35a52e-3c43-44b9-ae67-e1669d09b80e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:32.322706 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35a52e-3c43-44b9-ae67-e1669d09b80e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:32.323118 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e35a52e-3c43-44b9-ae67-e1669d09b80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:32.342015 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7e35a52e-3c43-44b9-ae67-e1669d09b80e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:32.343462 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7e35a52e-3c43-44b9-ae67-e1669d09b80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:32.374985 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c4de3-3641-4f0a-89de-f7177b3e401e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:32.376497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c4de3-3641-4f0a-89de-f7177b3e401e None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:32.426693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 181/345] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:32.428218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f83ed-e373-4c1c-9dd0-f0bfb5710714 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:32.435646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f83ed-e373-4c1c-9dd0-f0bfb5710714 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:32.436208 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7f83ed-e373-4c1c-9dd0-f0bfb5710714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:32.442913 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:32.466016 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7d7f83ed-e373-4c1c-9dd0-f0bfb5710714 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:32.467431 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7d7f83ed-e373-4c1c-9dd0-f0bfb5710714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:32.488350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 165/346] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:32.489278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c4de3-3641-4f0a-89de-f7177b3e401e None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:32.543321 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:32.574271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 182/347] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:32.575708 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f83ed-e373-4c1c-9dd0-f0bfb5710714 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:32.599608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c4de3-3641-4f0a-89de-f7177b3e401e None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:32.638376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bed93-51f9-4d42-abb9-41aae23fc13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:32.639101 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bed93-51f9-4d42-abb9-41aae23fc13d None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:32.639270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bed93-51f9-4d42-abb9-41aae23fc13d None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:32.639542 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bed93-51f9-4d42-abb9-41aae23fc13d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:32.675990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:32.722016 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bed93-51f9-4d42-abb9-41aae23fc13d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:32.723885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bed93-51f9-4d42-abb9-41aae23fc13d None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:32.789904 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7f83ed-e373-4c1c-9dd0-f0bfb5710714 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:32.834154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 166/348] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:32.840432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bed93-51f9-4d42-abb9-41aae23fc13d None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:32.852387 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd9279-4682-42f4-a01f-03f5e7eab51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:32.853071 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd9279-4682-42f4-a01f-03f5e7eab51e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:32.853435 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd9279-4682-42f4-a01f-03f5e7eab51e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:32.853774 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd9279-4682-42f4-a01f-03f5e7eab51e None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:32.855064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd9279-4682-42f4-a01f-03f5e7eab51e None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:32.856974 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd9279-4682-42f4-a01f-03f5e7eab51e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:32.858692 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 183/349] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:32 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:32.864103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd9279-4682-42f4-a01f-03f5e7eab51e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:32.911913 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:32.950628 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:32.955345 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bed93-51f9-4d42-abb9-41aae23fc13d None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:32.990187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db0f22-b153-4975-afea-74b1a1dd482a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:32.991111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db0f22-b153-4975-afea-74b1a1dd482a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:32.991566 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db0f22-b153-4975-afea-74b1a1dd482a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:32.992167 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db0f22-b153-4975-afea-74b1a1dd482a None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:32.993681 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db0f22-b153-4975-afea-74b1a1dd482a None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:32.995882 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db0f22-b153-4975-afea-74b1a1dd482a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:32.998206 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 167/350] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:32 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:32.999591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7907e8-acfa-454a-be20-7445421769e4 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.000152 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7907e8-acfa-454a-be20-7445421769e4 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.000613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7907e8-acfa-454a-be20-7445421769e4 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.002106 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 168/351] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:33.002986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7907e8-acfa-454a-be20-7445421769e4 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.013183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd9279-4682-42f4-a01f-03f5e7eab51e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:33.052624 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ee15e-94c1-46c5-a75e-28077fbfd0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:33.053265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ee15e-94c1-46c5-a75e-28077fbfd0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.053577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ee15e-94c1-46c5-a75e-28077fbfd0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.054091 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ee15e-94c1-46c5-a75e-28077fbfd0c4 None admin] PATH_INFO: `/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.061153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ee15e-94c1-46c5-a75e-28077fbfd0c4 None admin] RBAC: Authorizing `identity:get_group(group_id=0c59fa3bb1ec4933aec15e5f32612ef7)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:33.062848 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ee15e-94c1-46c5-a75e-28077fbfd0c4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:33.064537 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 184/352] 10.222.0.58 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:32 2022] GET /identity/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:33.066020 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d7dfa-ba20-4c86-879a-da59a48a3074 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.066141 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d7dfa-ba20-4c86-879a-da59a48a3074 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.066345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d7dfa-ba20-4c86-879a-da59a48a3074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.085041 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-440d7dfa-ba20-4c86-879a-da59a48a3074 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:33.086149 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-440d7dfa-ba20-4c86-879a-da59a48a3074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:33.121506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7907e8-acfa-454a-be20-7445421769e4 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:33.160343 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:33.169501 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d478a7-ce04-435b-b67b-5aa00196244d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:33.170480 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d478a7-ce04-435b-b67b-5aa00196244d None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.170906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d478a7-ce04-435b-b67b-5aa00196244d None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.171029 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 185/353] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:33.171426 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d478a7-ce04-435b-b67b-5aa00196244d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.171919 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d7dfa-ba20-4c86-879a-da59a48a3074 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.186544 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d478a7-ce04-435b-b67b-5aa00196244d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:33.191704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-36d478a7-ce04-435b-b67b-5aa00196244d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:33.191704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:33.191704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:33.191704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:33.191704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:33.191704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:33.191704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:33.191704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:33.191704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:33.191704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:33.191704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:33.191704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:33.191704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:33.191704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:33.191704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:33.191704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:33.192623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:33.192623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:33.192623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:33.192623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:33.192623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:33.192623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:33.192623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:33.192623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:33.192623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:33.192623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:33.192623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:33.192623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:33.192623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:33.192623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:33.192623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:33.192623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:33.193454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:33.193454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:33.193454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:33.193454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:33.193454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:33.193454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:33.193454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:33.193454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:33.193454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:33.193454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:33.193454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:33.193454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:33.197200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 169/354] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:33.198210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d478a7-ce04-435b-b67b-5aa00196244d None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.232094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:33.301322 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d7dfa-ba20-4c86-879a-da59a48a3074 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:33.314975 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d478a7-ce04-435b-b67b-5aa00196244d None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:33.349031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab1d9a-0fda-4df7-8382-17b3ba8f8be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:33.349886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab1d9a-0fda-4df7-8382-17b3ba8f8be0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.350432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab1d9a-0fda-4df7-8382-17b3ba8f8be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.350863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab1d9a-0fda-4df7-8382-17b3ba8f8be0 None admin] PATH_INFO: `/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.352795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab1d9a-0fda-4df7-8382-17b3ba8f8be0 None admin] RBAC: Authorizing `identity:get_group(group_id=0c59fa3bb1ec4933aec15e5f32612ef7)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:33.353442 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab1d9a-0fda-4df7-8382-17b3ba8f8be0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:33.359709 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 186/355] 10.222.0.58 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:33.359709 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab1d9a-0fda-4df7-8382-17b3ba8f8be0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.359940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e83891-b92c-4b88-99d5-f4feb22cbb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:33.359940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e83891-b92c-4b88-99d5-f4feb22cbb6d None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.359940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e83891-b92c-4b88-99d5-f4feb22cbb6d None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.359940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e83891-b92c-4b88-99d5-f4feb22cbb6d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.373422 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e83891-b92c-4b88-99d5-f4feb22cbb6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:33.378671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-27e83891-b92c-4b88-99d5-f4feb22cbb6d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:33.378671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:33.378671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:33.378671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:33.378671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:33.378671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:33.378671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:33.378671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:33.378671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:33.378671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:33.378671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:33.378671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:33.378671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:33.378671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:33.378671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:33.378671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:33.379551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:33.379551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:33.379551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:33.379551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:33.379551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:33.379551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:33.379551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:33.379551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:33.379551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:33.379551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:33.379551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:33.379551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:33.379551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:33.379551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:33.379551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:33.379551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:33.380433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:33.380433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:33.380433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:33.380433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:33.380433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:33.380433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:33.380433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:33.380433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:33.380433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:33.380433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:33.380433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 12:19:33.380433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:33.383522 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 170/356] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:33.384483 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e83891-b92c-4b88-99d5-f4feb22cbb6d None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.449944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:33.486181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab1d9a-0fda-4df7-8382-17b3ba8f8be0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:33.503866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:33.513628 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e83891-b92c-4b88-99d5-f4feb22cbb6d None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:33.525821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7848db2-6a60-4626-9f08-a3962257c499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:33.526430 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7848db2-6a60-4626-9f08-a3962257c499 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.527244 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7848db2-6a60-4626-9f08-a3962257c499 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.527518 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7848db2-6a60-4626-9f08-a3962257c499 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.528399 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7848db2-6a60-4626-9f08-a3962257c499 None admin] RBAC: Adding query filter params (group.id=0c59fa3bb1ec4933aec15e5f32612ef7, role.id=156b5bb1275a4dbeb61dbdade2c20d21, scope.project.id=c480d76c537348a7a86ae331f79d5e39) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:33.529150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7848db2-6a60-4626-9f08-a3962257c499 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:33.530594 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7848db2-6a60-4626-9f08-a3962257c499 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:33.539702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 187/357] 10.222.0.58 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/role_assignments?group.id=0c59fa3bb1ec4933aec15e5f32612ef7&scope.project.id=c480d76c537348a7a86ae331f79d5e39&role.id=156b5bb1275a4dbeb61dbdade2c20d21 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:33.541619 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e426e19-46a1-4d81-8635-030f95b06c52 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.542865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e426e19-46a1-4d81-8635-030f95b06c52 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.543591 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e426e19-46a1-4d81-8635-030f95b06c52 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.546059 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 188/358] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:33.547307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e426e19-46a1-4d81-8635-030f95b06c52 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.554257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da365b3-4312-44c1-81da-14979db42687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:33.554887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da365b3-4312-44c1-81da-14979db42687 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.555165 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da365b3-4312-44c1-81da-14979db42687 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.555435 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da365b3-4312-44c1-81da-14979db42687 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.556169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da365b3-4312-44c1-81da-14979db42687 None admin] RBAC: Adding query filter params (name=member) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:33.556477 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da365b3-4312-44c1-81da-14979db42687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:33.557961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da365b3-4312-44c1-81da-14979db42687 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:33.568326 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 171/359] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:33.569181 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da365b3-4312-44c1-81da-14979db42687 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.699055 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e426e19-46a1-4d81-8635-030f95b06c52 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:33.706540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da365b3-4312-44c1-81da-14979db42687 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:33.723522 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:33.744416 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194da2d-4119-450c-afd4-a4e4608c9dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:33.745908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194da2d-4119-450c-afd4-a4e4608c9dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.746203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194da2d-4119-450c-afd4-a4e4608c9dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.746572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f194da2d-4119-450c-afd4-a4e4608c9dd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.747727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194da2d-4119-450c-afd4-a4e4608c9dd3 None admin] RBAC: Adding query filter params (group.id=0c59fa3bb1ec4933aec15e5f32612ef7, role.id=4e002a22056744acb8c40a3236cab404, scope.project.id=c480d76c537348a7a86ae331f79d5e39) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:33.747727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194da2d-4119-450c-afd4-a4e4608c9dd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:33.749907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f194da2d-4119-450c-afd4-a4e4608c9dd3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:33.756867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 189/360] 10.222.0.58 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/role_assignments?group.id=0c59fa3bb1ec4933aec15e5f32612ef7&scope.project.id=c480d76c537348a7a86ae331f79d5e39&role.id=4e002a22056744acb8c40a3236cab404 => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:33.756867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194da2d-4119-450c-afd4-a4e4608c9dd3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.761446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf6834-94f4-403a-88e4-948719471eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:33.762590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf6834-94f4-403a-88e4-948719471eba None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.763085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf6834-94f4-403a-88e4-948719471eba None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.763671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf6834-94f4-403a-88e4-948719471eba None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.764575 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf6834-94f4-403a-88e4-948719471eba None admin] RBAC: Adding query filter params (name=reader) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:33.765090 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf6834-94f4-403a-88e4-948719471eba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:33.766732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf6834-94f4-403a-88e4-948719471eba None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:33.786692 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 172/361] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:33.788558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dfe9c-2719-4bc1-bf9c-bb8b69cab0a7 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.789050 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dfe9c-2719-4bc1-bf9c-bb8b69cab0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.789513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dfe9c-2719-4bc1-bf9c-bb8b69cab0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.816200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-220dfe9c-2719-4bc1-bf9c-bb8b69cab0a7 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:33.817753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-220dfe9c-2719-4bc1-bf9c-bb8b69cab0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:33.819607 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:33.898285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f194da2d-4119-450c-afd4-a4e4608c9dd3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:33.922769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 173/362] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:33.924107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dfe9c-2719-4bc1-bf9c-bb8b69cab0a7 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:33.940605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f355a43-1da8-4315-a163-6cef130cfc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:33.941388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f355a43-1da8-4315-a163-6cef130cfc15 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:33.941763 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f355a43-1da8-4315-a163-6cef130cfc15 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:33.942134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f355a43-1da8-4315-a163-6cef130cfc15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:33.956991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f355a43-1da8-4315-a163-6cef130cfc15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:33.962193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6f355a43-1da8-4315-a163-6cef130cfc15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:33.962193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:33.962193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:33.962193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:33.962193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:33.962193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:33.962193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:33.962193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:33.962193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:33.962193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:33.962193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:33.962193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:33.962193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:33.962193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:33.962193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:33.962193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:33.963291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:33.963291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:33.963291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:33.963291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:33.963291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:33.963291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:33.963291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:33.963291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:33.963291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:33.963291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:33.963291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:33.963291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:33.963291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:33.963291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:33.963291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:33.963291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:33.965125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:33.965125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:33.965125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:33.965125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:33.965125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:33.965125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:33.965125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:33.965125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:33.965125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:33.965125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:33.965125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:33.965125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:33.965125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 190/363] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:33.972684 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f355a43-1da8-4315-a163-6cef130cfc15 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:34.058700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dfe9c-2719-4bc1-bf9c-bb8b69cab0a7 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:34.066082 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:34.112003 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:34.115443 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd6819-43b9-43ab-a23c-77007819e744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:34.115577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f355a43-1da8-4315-a163-6cef130cfc15 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:34.116407 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd6819-43b9-43ab-a23c-77007819e744 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:34.116868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd6819-43b9-43ab-a23c-77007819e744 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:34.117487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd6819-43b9-43ab-a23c-77007819e744 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:34.160332 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2a7b9-ff38-40f3-a33a-275475cc3fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:34.161129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a7b9-ff38-40f3-a33a-275475cc3fcb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:34.161466 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a7b9-ff38-40f3-a33a-275475cc3fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:34.161799 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a7b9-ff38-40f3-a33a-275475cc3fcb None admin] PATH_INFO: `/v3/users/3de0b80a34824dd6a90b3b1fc64a0f58` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:34.163165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2a7b9-ff38-40f3-a33a-275475cc3fcb None admin] RBAC: Authorizing `identity:get_user(user_id=3de0b80a34824dd6a90b3b1fc64a0f58)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:34.164763 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2a7b9-ff38-40f3-a33a-275475cc3fcb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:34.166737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 191/364] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/users/3de0b80a34824dd6a90b3b1fc64a0f58 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:34.168508 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2a7b9-ff38-40f3-a33a-275475cc3fcb None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:34.211145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd6819-43b9-43ab-a23c-77007819e744 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:34.212842 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd6819-43b9-43ab-a23c-77007819e744 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:34.325836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2a7b9-ff38-40f3-a33a-275475cc3fcb None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:34.356264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 174/365] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:34.357228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd6819-43b9-43ab-a23c-77007819e744 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:34.360252 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:34.387699 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c1ac3-e0e9-4531-a031-44647f8bf8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:34.388883 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c1ac3-e0e9-4531-a031-44647f8bf8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:34.389241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c1ac3-e0e9-4531-a031-44647f8bf8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:34.389674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c1ac3-e0e9-4531-a031-44647f8bf8e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:34.390489 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c1ac3-e0e9-4531-a031-44647f8bf8e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:34.390850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c1ac3-e0e9-4531-a031-44647f8bf8e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:34.393470 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c1ac3-e0e9-4531-a031-44647f8bf8e7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:34.404769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 192/366] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:34.405486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c1ac3-e0e9-4531-a031-44647f8bf8e7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:34.443411 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:34.493645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd6819-43b9-43ab-a23c-77007819e744 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:34.531166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c1ac3-e0e9-4531-a031-44647f8bf8e7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:34.545833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc244f0-7f05-4e03-bf83-9aeddf00082b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:34.546508 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc244f0-7f05-4e03-bf83-9aeddf00082b None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:34.546793 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc244f0-7f05-4e03-bf83-9aeddf00082b None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:34.547065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc244f0-7f05-4e03-bf83-9aeddf00082b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:34.547828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc244f0-7f05-4e03-bf83-9aeddf00082b None admin] RBAC: Adding query filter params (role.id=0f1253d845894622b8091c95dfd079d3, user.id=3de0b80a34824dd6a90b3b1fc64a0f58, scope.system=all) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:34.548418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc244f0-7f05-4e03-bf83-9aeddf00082b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:34.550889 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc244f0-7f05-4e03-bf83-9aeddf00082b None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:34.566554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 175/367] 10.222.0.58 () {56 vars in 1308 bytes} [Wed Aug 24 12:19:34 2022] GET /identity/v3/role_assignments?user.id=3de0b80a34824dd6a90b3b1fc64a0f58&scope.system=all&role.id=0f1253d845894622b8091c95dfd079d3 => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:34.567540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc244f0-7f05-4e03-bf83-9aeddf00082b None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:34.600451 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c39c0-69a3-437a-bdd4-ffd27a089edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:34.601285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c39c0-69a3-437a-bdd4-ffd27a089edc None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:34.601757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c39c0-69a3-437a-bdd4-ffd27a089edc None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:34.602135 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c39c0-69a3-437a-bdd4-ffd27a089edc None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:34.607547 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c39c0-69a3-437a-bdd4-ffd27a089edc None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:34.609183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c39c0-69a3-437a-bdd4-ffd27a089edc None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:34.611024 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 193/368] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:34 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:34.611812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c39c0-69a3-437a-bdd4-ffd27a089edc None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:34.683846 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:34.692134 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc244f0-7f05-4e03-bf83-9aeddf00082b None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:34.774553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8d324-e5fe-450e-aaf4-34052838792b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:34.775111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8d324-e5fe-450e-aaf4-34052838792b None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:34.775389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8d324-e5fe-450e-aaf4-34052838792b None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:34.776189 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8d324-e5fe-450e-aaf4-34052838792b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:34.795226 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8d324-e5fe-450e-aaf4-34052838792b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:34.801817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-4cb8d324-e5fe-450e-aaf4-34052838792b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:34.801817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:34.801817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:34.801817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:34.801817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:34.801817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:34.801817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:34.801817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:34.801817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:34.801817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:34.801817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:34.801817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:34.801817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:34.801817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:34.801817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:34.801817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:34.802708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:34.802708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:34.802708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:34.802708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:34.802708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:34.802708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:34.802708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:34.802708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:34.802708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:34.802708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:34.802708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:34.802708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:34.802708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:34.802708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:34.802708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:34.802708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:34.803506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:34.803506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:34.803506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:34.803506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:34.803506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:34.803506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:34.803506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:34.803506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:34.803506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:34.803506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:34.803506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:34.803506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:34.803506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 176/369] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:34.804993 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8d324-e5fe-450e-aaf4-34052838792b None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:34.823270 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:34.830527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c39c0-69a3-437a-bdd4-ffd27a089edc None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:34.881460 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c2db6-5bc8-4978-b4c0-b30cca3b44a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:34.881460 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c2db6-5bc8-4978-b4c0-b30cca3b44a1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:34.881745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c2db6-5bc8-4978-b4c0-b30cca3b44a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:34.881939 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c2db6-5bc8-4978-b4c0-b30cca3b44a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:34.952793 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8d324-e5fe-450e-aaf4-34052838792b None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:34.976934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c2db6-5bc8-4978-b4c0-b30cca3b44a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:34.978591 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c2db6-5bc8-4978-b4c0-b30cca3b44a1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:34.988720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:35.007284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28710fc-b649-42e1-9501-a4846af091c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:35.008068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28710fc-b649-42e1-9501-a4846af091c4 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:35.008353 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28710fc-b649-42e1-9501-a4846af091c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:35.008627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28710fc-b649-42e1-9501-a4846af091c4 None admin] PATH_INFO: `/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:35.010331 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28710fc-b649-42e1-9501-a4846af091c4 None admin] RBAC: Authorizing `identity:get_group(group_id=0c59fa3bb1ec4933aec15e5f32612ef7)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:35.011844 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28710fc-b649-42e1-9501-a4846af091c4 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:35.014074 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 177/370] 10.222.0.58 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:34 2022] GET /identity/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:35.016267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28710fc-b649-42e1-9501-a4846af091c4 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:35.111390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:35.118308 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 194/371] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:35.121126 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c2db6-5bc8-4978-b4c0-b30cca3b44a1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:35.155484 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28710fc-b649-42e1-9501-a4846af091c4 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:35.192795 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7f9f6-993a-4a1f-9170-8fb620c06f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:35.193322 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7f9f6-993a-4a1f-9170-8fb620c06f28 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:35.193606 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7f9f6-993a-4a1f-9170-8fb620c06f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:35.193879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7f9f6-993a-4a1f-9170-8fb620c06f28 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:35.194599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7f9f6-993a-4a1f-9170-8fb620c06f28 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:35.194938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7f9f6-993a-4a1f-9170-8fb620c06f28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:35.196743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed7f9f6-993a-4a1f-9170-8fb620c06f28 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:35.207901 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 178/372] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:35.208731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7f9f6-993a-4a1f-9170-8fb620c06f28 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:35.245087 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c2db6-5bc8-4978-b4c0-b30cca3b44a1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:35.264866 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:35.285604 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27a452-4d13-4c26-9f11-cee69b70807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:35.286440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27a452-4d13-4c26-9f11-cee69b70807d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:35.286861 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27a452-4d13-4c26-9f11-cee69b70807d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:35.287274 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27a452-4d13-4c26-9f11-cee69b70807d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:35.288203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27a452-4d13-4c26-9f11-cee69b70807d None admin] RBAC: Adding query filter params (group.id=0c59fa3bb1ec4933aec15e5f32612ef7, role.id=156b5bb1275a4dbeb61dbdade2c20d21, scope.project.id=d9b6194f22b34cd695de9f04dace7e99) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:35.288654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27a452-4d13-4c26-9f11-cee69b70807d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:35.290357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27a452-4d13-4c26-9f11-cee69b70807d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:35.297632 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 195/373] 10.222.0.58 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:35 2022] GET /identity/v3/role_assignments?group.id=0c59fa3bb1ec4933aec15e5f32612ef7&scope.project.id=d9b6194f22b34cd695de9f04dace7e99&role.id=156b5bb1275a4dbeb61dbdade2c20d21 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:35.298805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27a452-4d13-4c26-9f11-cee69b70807d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:35.318028 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7f9f6-993a-4a1f-9170-8fb620c06f28 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:35.358258 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123085d7-1e97-401f-8a5c-eb7fa9b5884f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:35.358914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123085d7-1e97-401f-8a5c-eb7fa9b5884f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:35.359185 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123085d7-1e97-401f-8a5c-eb7fa9b5884f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:35.359460 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123085d7-1e97-401f-8a5c-eb7fa9b5884f None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:35.360848 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123085d7-1e97-401f-8a5c-eb7fa9b5884f None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:35.362478 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123085d7-1e97-401f-8a5c-eb7fa9b5884f None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:35.364154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 179/374] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:35 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:35.369899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123085d7-1e97-401f-8a5c-eb7fa9b5884f None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:35.375043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:35.430320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27a452-4d13-4c26-9f11-cee69b70807d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:35.474921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d7e70-4f80-4db0-a9f8-09c596512374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:35.476009 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d7e70-4f80-4db0-a9f8-09c596512374 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:35.476492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d7e70-4f80-4db0-a9f8-09c596512374 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:35.476963 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d7e70-4f80-4db0-a9f8-09c596512374 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:35.520870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123085d7-1e97-401f-8a5c-eb7fa9b5884f None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:35.554044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:35.575839 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27244cf0-eb88-4ec7-9aae-a88b1c801f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:35.576040 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d7e70-4f80-4db0-a9f8-09c596512374 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:35.576928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27244cf0-eb88-4ec7-9aae-a88b1c801f36 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:35.577469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d7e70-4f80-4db0-a9f8-09c596512374 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:35.577722 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27244cf0-eb88-4ec7-9aae-a88b1c801f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:35.578234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27244cf0-eb88-4ec7-9aae-a88b1c801f36 None admin] PATH_INFO: `/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:35.579836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27244cf0-eb88-4ec7-9aae-a88b1c801f36 None admin] RBAC: Authorizing `identity:get_group(group_id=0c59fa3bb1ec4933aec15e5f32612ef7)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:35.582407 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27244cf0-eb88-4ec7-9aae-a88b1c801f36 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:35.584556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 180/375] 10.222.0.58 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:35 2022] GET /identity/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:35.592028 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27244cf0-eb88-4ec7-9aae-a88b1c801f36 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:35.661023 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:35.713589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 196/376] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:35.720625 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d7e70-4f80-4db0-a9f8-09c596512374 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:35.733005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27244cf0-eb88-4ec7-9aae-a88b1c801f36 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:35.778293 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eef5e0-3d92-4287-8f4b-27032e1dada6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:35.778915 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eef5e0-3d92-4287-8f4b-27032e1dada6 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:35.779192 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eef5e0-3d92-4287-8f4b-27032e1dada6 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:35.779461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eef5e0-3d92-4287-8f4b-27032e1dada6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:35.780325 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eef5e0-3d92-4287-8f4b-27032e1dada6 None admin] RBAC: Adding query filter params (group.id=0c59fa3bb1ec4933aec15e5f32612ef7, role.id=4e002a22056744acb8c40a3236cab404, scope.project.id=d9b6194f22b34cd695de9f04dace7e99) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:35.780623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eef5e0-3d92-4287-8f4b-27032e1dada6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:35.782141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eef5e0-3d92-4287-8f4b-27032e1dada6 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:35.788455 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 181/377] 10.222.0.58 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:35 2022] GET /identity/v3/role_assignments?group.id=0c59fa3bb1ec4933aec15e5f32612ef7&scope.project.id=d9b6194f22b34cd695de9f04dace7e99&role.id=4e002a22056744acb8c40a3236cab404 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:35.842940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d7e70-4f80-4db0-a9f8-09c596512374 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:35.890381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d1c7a-e3ed-4520-a8cc-b60afcb42a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:35.891390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d1c7a-e3ed-4520-a8cc-b60afcb42a31 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:35.891892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d1c7a-e3ed-4520-a8cc-b60afcb42a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:35.892364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d1c7a-e3ed-4520-a8cc-b60afcb42a31 None admin] PATH_INFO: `/v3/projects/ae0194ad9d2f4f1ebdd6b4e39c18be2c` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:35.896396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d1c7a-e3ed-4520-a8cc-b60afcb42a31 None admin] RBAC: Authorizing `identity:get_project(project_id=ae0194ad9d2f4f1ebdd6b4e39c18be2c)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:35.898224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d1c7a-e3ed-4520-a8cc-b60afcb42a31 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:35.900277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 197/378] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:35 2022] GET /identity/v3/projects/ae0194ad9d2f4f1ebdd6b4e39c18be2c => generated 344 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:35.906973 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eef5e0-3d92-4287-8f4b-27032e1dada6 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:35.980753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:36.046306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eef5e0-3d92-4287-8f4b-27032e1dada6 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:36.096591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651fc9d-d9ad-4b70-b7a6-ec963c1a6ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:36.097275 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651fc9d-d9ad-4b70-b7a6-ec963c1a6ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:36.097562 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651fc9d-d9ad-4b70-b7a6-ec963c1a6ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:36.097839 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651fc9d-d9ad-4b70-b7a6-ec963c1a6ba4 None admin] PATH_INFO: `/v3/groups/875fe1a6a833450ebf5265f9ffba0ea8` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:36.110406 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651fc9d-d9ad-4b70-b7a6-ec963c1a6ba4 None admin] RBAC: Authorizing `identity:get_group(group_id=875fe1a6a833450ebf5265f9ffba0ea8)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:36.111982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651fc9d-d9ad-4b70-b7a6-ec963c1a6ba4 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:36.113682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 182/379] 10.222.0.58 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:35 2022] GET /identity/v3/groups/875fe1a6a833450ebf5265f9ffba0ea8 => generated 229 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:36.119672 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d1c7a-e3ed-4520-a8cc-b60afcb42a31 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:36.167449 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:36.244086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f2143-81fb-4696-b205-04591ce2a0c3 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:36.244383 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f2143-81fb-4696-b205-04591ce2a0c3 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:36.244608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f2143-81fb-4696-b205-04591ce2a0c3 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:36.246292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 183/380] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:36.254777 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b552e1f-1dcf-4764-9f83-ea279ede8132 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:36.254777 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b552e1f-1dcf-4764-9f83-ea279ede8132 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:36.254777 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b552e1f-1dcf-4764-9f83-ea279ede8132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:36.259647 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d1c7a-e3ed-4520-a8cc-b60afcb42a31 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:36.279164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-7b552e1f-1dcf-4764-9f83-ea279ede8132 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:36.281367 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7b552e1f-1dcf-4764-9f83-ea279ede8132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:36.323196 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdb330-62f0-453e-8e3e-d06e2815f56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:36.323991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdb330-62f0-453e-8e3e-d06e2815f56b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:36.324467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdb330-62f0-453e-8e3e-d06e2815f56b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:36.325173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdb330-62f0-453e-8e3e-d06e2815f56b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:36.326075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdb330-62f0-453e-8e3e-d06e2815f56b None admin] RBAC: Adding query filter params (group.id=875fe1a6a833450ebf5265f9ffba0ea8, role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=ae0194ad9d2f4f1ebdd6b4e39c18be2c) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:36.326386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdb330-62f0-453e-8e3e-d06e2815f56b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:36.327958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdb330-62f0-453e-8e3e-d06e2815f56b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:36.338470 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 198/381] 10.222.0.58 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:36 2022] GET /identity/v3/role_assignments?group.id=875fe1a6a833450ebf5265f9ffba0ea8&scope.project.id=ae0194ad9d2f4f1ebdd6b4e39c18be2c&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:36.367063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 184/382] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:36.381511 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0dac8-90a2-4f37-b8a8-e197bc32f211 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:36.381799 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0dac8-90a2-4f37-b8a8-e197bc32f211 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:36.382082 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0dac8-90a2-4f37-b8a8-e197bc32f211 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:36.383791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 199/383] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:36.390101 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac64ba-dbce-4962-901a-ffe29a52c100 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:36.390664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac64ba-dbce-4962-901a-ffe29a52c100 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:36.391167 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac64ba-dbce-4962-901a-ffe29a52c100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:36.411904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-31ac64ba-dbce-4962-901a-ffe29a52c100 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:36.413478 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-31ac64ba-dbce-4962-901a-ffe29a52c100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:36.468023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:36.497538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 185/384] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:36.757858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0dac8-90a2-4f37-b8a8-e197bc32f211 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:36.784818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac64ba-dbce-4962-901a-ffe29a52c100 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:36.927729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:36.934969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0dac8-90a2-4f37-b8a8-e197bc32f211 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:36.976222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7124b9d-bb86-4a6f-a241-07d4bdc27177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:36.976865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7124b9d-bb86-4a6f-a241-07d4bdc27177 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:36.977150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7124b9d-bb86-4a6f-a241-07d4bdc27177 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:36.977424 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7124b9d-bb86-4a6f-a241-07d4bdc27177 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:36.993732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7124b9d-bb86-4a6f-a241-07d4bdc27177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:36.999652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a7124b9d-bb86-4a6f-a241-07d4bdc27177 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:36.999652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:36.999652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:36.999652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:36.999652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:36.999652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:36.999652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:36.999652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:36.999652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:36.999652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:36.999652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:36.999652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:36.999652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:36.999652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:36.999652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:36.999652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:37.000546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:37.000546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:37.000546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:37.000546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:37.000546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:37.000546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:37.000546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:37.000546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:37.000546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:37.000546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:37.000546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:37.000546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:37.000546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:37.000546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:37.000546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:37.000546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:37.001369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:37.001369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:37.001369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:37.001369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:37.001369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:37.001369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:37.001369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:37.001369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:37.001369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:37.001369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:37.001369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:37.001369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:37.001369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 200/385] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:37.007200 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7124b9d-bb86-4a6f-a241-07d4bdc27177 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:37.027429 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac64ba-dbce-4962-901a-ffe29a52c100 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:37.099718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663be15f-9d50-46dd-a5ce-b067bfd035cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:37.100567 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663be15f-9d50-46dd-a5ce-b067bfd035cf None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:37.100901 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663be15f-9d50-46dd-a5ce-b067bfd035cf None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:37.101251 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663be15f-9d50-46dd-a5ce-b067bfd035cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:37.119066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663be15f-9d50-46dd-a5ce-b067bfd035cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:37.124593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-663be15f-9d50-46dd-a5ce-b067bfd035cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:37.124593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:37.124593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:37.124593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:37.124593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:37.124593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:37.124593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:37.124593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:37.124593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:37.124593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:37.124593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:37.124593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:37.124593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:37.124593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:37.124593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:37.124593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:37.125513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:37.125513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:37.125513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:37.125513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:37.125513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:37.125513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:37.125513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:37.125513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:37.125513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:37.125513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:37.125513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:37.125513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:37.125513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:37.125513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:37.125513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:37.125513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:37.126309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:37.126309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:37.126309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:37.126309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:37.126309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:37.126309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:37.126309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:37.126309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:37.126309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:37.126309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:37.126309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:37.126309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:37.126309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 186/386] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:37.134497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663be15f-9d50-46dd-a5ce-b067bfd035cf None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:37.153367 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7124b9d-bb86-4a6f-a241-07d4bdc27177 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:37.200430 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948542ce-5aab-4361-88bc-115713e798dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:37.201336 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948542ce-5aab-4361-88bc-115713e798dd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:37.201840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948542ce-5aab-4361-88bc-115713e798dd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:37.202314 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948542ce-5aab-4361-88bc-115713e798dd None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:37.203146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948542ce-5aab-4361-88bc-115713e798dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:37.203643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948542ce-5aab-4361-88bc-115713e798dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:37.205584 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948542ce-5aab-4361-88bc-115713e798dd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:37.216722 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:37.218874 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 201/387] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:37.221947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948542ce-5aab-4361-88bc-115713e798dd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:37.261117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:37.316939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663be15f-9d50-46dd-a5ce-b067bfd035cf None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:37.366652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4af5f9-4f07-4e70-8c13-4801add46dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:37.367430 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4af5f9-4f07-4e70-8c13-4801add46dff None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:37.367865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4af5f9-4f07-4e70-8c13-4801add46dff None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:37.368239 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4af5f9-4f07-4e70-8c13-4801add46dff None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:37.369009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4af5f9-4f07-4e70-8c13-4801add46dff None admin] RBAC: Adding query filter params (name=member) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:37.369416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4af5f9-4f07-4e70-8c13-4801add46dff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:37.371105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4af5f9-4f07-4e70-8c13-4801add46dff None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:37.379328 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948542ce-5aab-4361-88bc-115713e798dd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:37.380603 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 187/388] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:37.386577 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4af5f9-4f07-4e70-8c13-4801add46dff None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:37.427148 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b5a63-7d71-4f64-b7a9-16267445ae0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:37.428585 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b5a63-7d71-4f64-b7a9-16267445ae0c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:37.428585 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b5a63-7d71-4f64-b7a9-16267445ae0c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:37.428585 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b5a63-7d71-4f64-b7a9-16267445ae0c None admin] PATH_INFO: `/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:37.429879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b5a63-7d71-4f64-b7a9-16267445ae0c None admin] RBAC: Authorizing `identity:get_group(group_id=0c59fa3bb1ec4933aec15e5f32612ef7)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:37.431428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b5a63-7d71-4f64-b7a9-16267445ae0c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:37.434393 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 202/389] 10.222.0.58 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:37 2022] GET /identity/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:37.439781 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b5a63-7d71-4f64-b7a9-16267445ae0c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:37.505189 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:37.510103 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4af5f9-4f07-4e70-8c13-4801add46dff None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:37.549999 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454cf44-2fcb-4507-b922-eaa4352fbd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:37.550978 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454cf44-2fcb-4507-b922-eaa4352fbd7c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:37.551436 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454cf44-2fcb-4507-b922-eaa4352fbd7c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:37.551882 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8454cf44-2fcb-4507-b922-eaa4352fbd7c None admin] PATH_INFO: `/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:37.553426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:37.553727 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454cf44-2fcb-4507-b922-eaa4352fbd7c None admin] RBAC: Authorizing `identity:get_group(group_id=0c59fa3bb1ec4933aec15e5f32612ef7)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:37.556012 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8454cf44-2fcb-4507-b922-eaa4352fbd7c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:37.558314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 188/390] 10.222.0.58 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:37 2022] GET /identity/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:37.564768 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454cf44-2fcb-4507-b922-eaa4352fbd7c None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:37.577306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b5a63-7d71-4f64-b7a9-16267445ae0c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:37.620215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f09e72-a46e-437e-be88-98f41e033529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:37.621003 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f09e72-a46e-437e-be88-98f41e033529 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:37.621378 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f09e72-a46e-437e-be88-98f41e033529 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:37.621624 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f09e72-a46e-437e-be88-98f41e033529 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:37.622892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f09e72-a46e-437e-be88-98f41e033529 None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:37.624554 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f09e72-a46e-437e-be88-98f41e033529 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:37.626591 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 203/391] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:37 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:37.632050 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f09e72-a46e-437e-be88-98f41e033529 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:37.696122 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8454cf44-2fcb-4507-b922-eaa4352fbd7c None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:37.734337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5390ba7-62b9-4b61-91ef-eb33821b0c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:37.736068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5390ba7-62b9-4b61-91ef-eb33821b0c5e None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:37.736668 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5390ba7-62b9-4b61-91ef-eb33821b0c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:37.741468 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5390ba7-62b9-4b61-91ef-eb33821b0c5e None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:37.743071 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5390ba7-62b9-4b61-91ef-eb33821b0c5e None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:37.745635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5390ba7-62b9-4b61-91ef-eb33821b0c5e None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:37.748176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 189/392] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:37 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:37.756450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5390ba7-62b9-4b61-91ef-eb33821b0c5e None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:37.799712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:37.865904 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f09e72-a46e-437e-be88-98f41e033529 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:37.881805 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5390ba7-62b9-4b61-91ef-eb33821b0c5e None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:37.923041 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245a1d2-2162-4842-8b8a-d0d24a050b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:37.924107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245a1d2-2162-4842-8b8a-d0d24a050b10 None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:37.925013 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245a1d2-2162-4842-8b8a-d0d24a050b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:37.925491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245a1d2-2162-4842-8b8a-d0d24a050b10 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39/groups/0c59fa3bb1ec4933aec15e5f32612ef7/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:37.928526 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245a1d2-2162-4842-8b8a-d0d24a050b10 None admin] RBAC: Authorizing `identity:create_grant(project_id=c480d76c537348a7a86ae331f79d5e39, group_id=0c59fa3bb1ec4933aec15e5f32612ef7, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:37.930172 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245a1d2-2162-4842-8b8a-d0d24a050b10 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:37.947790 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 190/393] 10.222.0.58 () {58 vars in 1442 bytes} [Wed Aug 24 12:19:37 2022] PUT /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39/groups/0c59fa3bb1ec4933aec15e5f32612ef7/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:37.959739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a983b-8729-4a3d-9ac5-4df1efc57c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:37.960841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a983b-8729-4a3d-9ac5-4df1efc57c15 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:37.961323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a983b-8729-4a3d-9ac5-4df1efc57c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:37.961792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a983b-8729-4a3d-9ac5-4df1efc57c15 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39/groups/0c59fa3bb1ec4933aec15e5f32612ef7/roles/4e002a22056744acb8c40a3236cab404` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:37.968024 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a983b-8729-4a3d-9ac5-4df1efc57c15 None admin] RBAC: Authorizing `identity:create_grant(project_id=c480d76c537348a7a86ae331f79d5e39, group_id=0c59fa3bb1ec4933aec15e5f32612ef7, role_id=4e002a22056744acb8c40a3236cab404)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:37.970149 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a983b-8729-4a3d-9ac5-4df1efc57c15 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:38.000461 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 204/394] 10.222.0.58 () {58 vars in 1442 bytes} [Wed Aug 24 12:19:37 2022] PUT /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39/groups/0c59fa3bb1ec4933aec15e5f32612ef7/roles/4e002a22056744acb8c40a3236cab404 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:38.123931 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784804a-5de7-425e-ae18-2234e4630276 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:38.124543 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784804a-5de7-425e-ae18-2234e4630276 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:38.125023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784804a-5de7-425e-ae18-2234e4630276 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:38.126522 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 191/395] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:38.132319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c635a6-0486-4c38-9e50-9be95f9f75e4 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:38.132895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c635a6-0486-4c38-9e50-9be95f9f75e4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:38.133389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c635a6-0486-4c38-9e50-9be95f9f75e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:38.154203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-42c635a6-0486-4c38-9e50-9be95f9f75e4 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:38.155519 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-42c635a6-0486-4c38-9e50-9be95f9f75e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:38.165223 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:38.241969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 205/396] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:38.432131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c5b10-79f7-4910-8ced-1353b22c5943 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:38.432628 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c5b10-79f7-4910-8ced-1353b22c5943 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:38.432628 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c5b10-79f7-4910-8ced-1353b22c5943 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:38.434872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 192/397] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:38.440053 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804f694-24fd-419e-a2c7-bd01fd6de421 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:38.440418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804f694-24fd-419e-a2c7-bd01fd6de421 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:38.440764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804f694-24fd-419e-a2c7-bd01fd6de421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:38.459973 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b804f694-24fd-419e-a2c7-bd01fd6de421 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:38.461158 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b804f694-24fd-419e-a2c7-bd01fd6de421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:38.533062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c5b10-79f7-4910-8ced-1353b22c5943 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:38.535677 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 206/398] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:38.643127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:38.651384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c5b10-79f7-4910-8ced-1353b22c5943 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:38.691546 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f87ac1-ae07-4c7a-8555-61ecf0e29d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:38.692143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f87ac1-ae07-4c7a-8555-61ecf0e29d11 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:38.692427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f87ac1-ae07-4c7a-8555-61ecf0e29d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:38.692758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f87ac1-ae07-4c7a-8555-61ecf0e29d11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:38.704451 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f87ac1-ae07-4c7a-8555-61ecf0e29d11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:38.709469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-40f87ac1-ae07-4c7a-8555-61ecf0e29d11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:38.709469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:38.709469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:38.709469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:38.709469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:38.709469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:38.709469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:38.709469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:38.709469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:38.709469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:38.709469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:38.709469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:38.709469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:38.709469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:38.709469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:38.709469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:38.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:38.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:38.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:38.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:38.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:38.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:38.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:38.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:38.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:38.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:38.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:38.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:38.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:38.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:38.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:38.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:38.713452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:38.713452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:38.713452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:38.713452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:38.713452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:38.713452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:38.713452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:38.713452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:38.713452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:38.713452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:38.713452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:38.713452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:38.713452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 193/399] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:38.724736 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804f694-24fd-419e-a2c7-bd01fd6de421 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:38.802140 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:38.835457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804f694-24fd-419e-a2c7-bd01fd6de421 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:38.872978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed705c-6001-4972-9427-9ba6a481a445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:38.876611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed705c-6001-4972-9427-9ba6a481a445 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:38.876611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed705c-6001-4972-9427-9ba6a481a445 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:38.876611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed705c-6001-4972-9427-9ba6a481a445 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:38.876611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed705c-6001-4972-9427-9ba6a481a445 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:38.876611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed705c-6001-4972-9427-9ba6a481a445 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:38.880249 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f87ac1-ae07-4c7a-8555-61ecf0e29d11 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:38.880402 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed705c-6001-4972-9427-9ba6a481a445 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:38.892342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 207/400] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:38.893154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed705c-6001-4972-9427-9ba6a481a445 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:39.004355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed705c-6001-4972-9427-9ba6a481a445 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:39.043460 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f87ac1-ae07-4c7a-8555-61ecf0e29d11 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:39.071161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c64ca-d42e-4586-8f1d-423aeae115fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:39.071829 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c64ca-d42e-4586-8f1d-423aeae115fa None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:39.072113 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c64ca-d42e-4586-8f1d-423aeae115fa None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:39.072390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c64ca-d42e-4586-8f1d-423aeae115fa None admin] PATH_INFO: `/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:39.075672 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c64ca-d42e-4586-8f1d-423aeae115fa None admin] RBAC: Authorizing `identity:get_group(group_id=0c59fa3bb1ec4933aec15e5f32612ef7)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:39.077867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c64ca-d42e-4586-8f1d-423aeae115fa None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:39.087111 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 208/401] 10.222.0.58 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:38 2022] GET /identity/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:39.097879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c64ca-d42e-4586-8f1d-423aeae115fa None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:39.102349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:39.107314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625fcd5-702a-4d46-b86c-344e31558a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:39.108231 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625fcd5-702a-4d46-b86c-344e31558a77 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:39.108699 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625fcd5-702a-4d46-b86c-344e31558a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:39.109138 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625fcd5-702a-4d46-b86c-344e31558a77 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:39.119680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625fcd5-702a-4d46-b86c-344e31558a77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:39.125043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-f625fcd5-702a-4d46-b86c-344e31558a77 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:39.125043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:39.125043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:39.125043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:39.125043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:39.125043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:39.125043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:39.125043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:39.125043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:39.125043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:39.125043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:39.125043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:39.125043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:39.125043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:39.125043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:39.125043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:39.125906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:39.125906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:39.125906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:39.125906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:39.125906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:39.125906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:39.125906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:39.125906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:39.125906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:39.125906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:39.125906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:39.125906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:39.125906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:39.125906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:39.125906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:39.125906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:39.126695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:39.126695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:39.126695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:39.126695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:39.126695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:39.126695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:39.126695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:39.126695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:39.126695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:39.126695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:39.126695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:39.126695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:39.133723 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 194/402] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:39.135112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e7c0e-136b-410a-b3c9-aff3c3279f85 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:39.135564 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e7c0e-136b-410a-b3c9-aff3c3279f85 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:39.136567 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751e7c0e-136b-410a-b3c9-aff3c3279f85 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:39.142549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 195/403] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:39.143500 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e7c0e-136b-410a-b3c9-aff3c3279f85 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:39.160804 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:39.221929 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c64ca-d42e-4586-8f1d-423aeae115fa None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:39.250843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751e7c0e-136b-410a-b3c9-aff3c3279f85 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:39.258812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef85ddf-e2cd-4e22-8eaf-95260c327002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:39.259480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef85ddf-e2cd-4e22-8eaf-95260c327002 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:39.260202 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef85ddf-e2cd-4e22-8eaf-95260c327002 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:39.260202 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef85ddf-e2cd-4e22-8eaf-95260c327002 None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:39.261967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef85ddf-e2cd-4e22-8eaf-95260c327002 None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:39.263648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef85ddf-e2cd-4e22-8eaf-95260c327002 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:39.270310 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 209/404] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:39 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:39.273617 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f61143-55e7-4a8d-bd4b-958b6b3f14d0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:39.273617 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f61143-55e7-4a8d-bd4b-958b6b3f14d0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:39.273617 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f61143-55e7-4a8d-bd4b-958b6b3f14d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:39.293741 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a1c7e-81f0-489e-9e08-cff0e22df8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:39.294196 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c2f61143-55e7-4a8d-bd4b-958b6b3f14d0 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:39.294874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a1c7e-81f0-489e-9e08-cff0e22df8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:39.295333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a1c7e-81f0-489e-9e08-cff0e22df8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:39.295561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c2f61143-55e7-4a8d-bd4b-958b6b3f14d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:39.295989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a1c7e-81f0-489e-9e08-cff0e22df8a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:39.296822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a1c7e-81f0-489e-9e08-cff0e22df8a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:39.297305 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a1c7e-81f0-489e-9e08-cff0e22df8a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:39.298857 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a1c7e-81f0-489e-9e08-cff0e22df8a3 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:39.311740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 196/405] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:39.312852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a1c7e-81f0-489e-9e08-cff0e22df8a3 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:39.366886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:39.379647 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 210/406] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:39.379647 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f61143-55e7-4a8d-bd4b-958b6b3f14d0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:39.418301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:39.453305 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a1c7e-81f0-489e-9e08-cff0e22df8a3 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:39.487045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f61143-55e7-4a8d-bd4b-958b6b3f14d0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:39.527980 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b41d5e-4c05-480f-9908-9ec8f2da9509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:39.528540 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b41d5e-4c05-480f-9908-9ec8f2da9509 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:39.528819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b41d5e-4c05-480f-9908-9ec8f2da9509 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:39.529091 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b41d5e-4c05-480f-9908-9ec8f2da9509 None admin] PATH_INFO: `/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:39.530199 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b41d5e-4c05-480f-9908-9ec8f2da9509 None admin] RBAC: Authorizing `identity:get_group(group_id=0c59fa3bb1ec4933aec15e5f32612ef7)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:39.531663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b41d5e-4c05-480f-9908-9ec8f2da9509 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:39.533368 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 211/407] 10.222.0.58 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:39 2022] GET /identity/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:39.538891 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b41d5e-4c05-480f-9908-9ec8f2da9509 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:39.540404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beadd496-2632-4e95-8921-c8e0d5e72f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:39.541331 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadd496-2632-4e95-8921-c8e0d5e72f85 None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:39.541815 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadd496-2632-4e95-8921-c8e0d5e72f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:39.542344 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadd496-2632-4e95-8921-c8e0d5e72f85 None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99/groups/0c59fa3bb1ec4933aec15e5f32612ef7/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:39.544983 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beadd496-2632-4e95-8921-c8e0d5e72f85 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b6194f22b34cd695de9f04dace7e99, group_id=0c59fa3bb1ec4933aec15e5f32612ef7, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:39.546739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beadd496-2632-4e95-8921-c8e0d5e72f85 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:39.595440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 197/408] 10.222.0.58 () {58 vars in 1442 bytes} [Wed Aug 24 12:19:39 2022] PUT /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99/groups/0c59fa3bb1ec4933aec15e5f32612ef7/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:39.620090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:39.658519 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b41d5e-4c05-480f-9908-9ec8f2da9509 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:39.696713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774a163-a577-4049-b079-121687589b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:39.697367 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774a163-a577-4049-b079-121687589b2c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:39.697645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774a163-a577-4049-b079-121687589b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:39.697916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6774a163-a577-4049-b079-121687589b2c None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:39.699140 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774a163-a577-4049-b079-121687589b2c None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:39.700953 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6774a163-a577-4049-b079-121687589b2c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:39.702916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 212/409] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:39 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:39.712742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beadd496-2632-4e95-8921-c8e0d5e72f85 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:39.860388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774a163-a577-4049-b079-121687589b2c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:39.867814 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beadd496-2632-4e95-8921-c8e0d5e72f85 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:39.910632 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129967f-5786-43ff-b78e-d1a322f4dd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:39.911306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129967f-5786-43ff-b78e-d1a322f4dd9a None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:39.911585 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129967f-5786-43ff-b78e-d1a322f4dd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:39.911882 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129967f-5786-43ff-b78e-d1a322f4dd9a None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99/groups/0c59fa3bb1ec4933aec15e5f32612ef7/roles/4e002a22056744acb8c40a3236cab404` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:39.915438 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129967f-5786-43ff-b78e-d1a322f4dd9a None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b6194f22b34cd695de9f04dace7e99, group_id=0c59fa3bb1ec4933aec15e5f32612ef7, role_id=4e002a22056744acb8c40a3236cab404)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:39.916925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129967f-5786-43ff-b78e-d1a322f4dd9a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:39.921722 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:39.936706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 198/410] 10.222.0.58 () {58 vars in 1442 bytes} [Wed Aug 24 12:19:39 2022] PUT /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99/groups/0c59fa3bb1ec4933aec15e5f32612ef7/roles/4e002a22056744acb8c40a3236cab404 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:40.043012 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6774a163-a577-4049-b079-121687589b2c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:40.096481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c064b3b-4dd9-402a-899b-17d19c39cc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:40.097831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c064b3b-4dd9-402a-899b-17d19c39cc5b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.098575 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c064b3b-4dd9-402a-899b-17d19c39cc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.099109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c064b3b-4dd9-402a-899b-17d19c39cc5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.102626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:40.120306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c064b3b-4dd9-402a-899b-17d19c39cc5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:40.125689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-1c064b3b-4dd9-402a-899b-17d19c39cc5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:40.125689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:40.125689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:40.125689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:40.125689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:40.125689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:40.125689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:40.125689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:40.125689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:40.125689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:40.125689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:40.125689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:40.125689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:40.125689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:40.125689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:40.125689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:40.126831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:40.126831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:40.126831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:40.126831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:40.126831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:40.126831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:40.126831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:40.126831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:40.126831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:40.126831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:40.126831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:40.126831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:40.126831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:40.126831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:40.126831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:40.126831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:40.127892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:40.127892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:40.127892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:40.127892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:40.127892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:40.127892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:40.127892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:40.127892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:40.127892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:40.127892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:40.127892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:40.127892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:40.129957 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 213/411] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:40.136451 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129967f-5786-43ff-b78e-d1a322f4dd9a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:40.249140 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129967f-5786-43ff-b78e-d1a322f4dd9a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:40.285474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275d8ed-7773-40b7-bef9-90b62c99c051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:40.286352 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275d8ed-7773-40b7-bef9-90b62c99c051 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.286785 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275d8ed-7773-40b7-bef9-90b62c99c051 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.287227 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275d8ed-7773-40b7-bef9-90b62c99c051 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.288106 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275d8ed-7773-40b7-bef9-90b62c99c051 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:40.288553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275d8ed-7773-40b7-bef9-90b62c99c051 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:40.290144 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275d8ed-7773-40b7-bef9-90b62c99c051 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:40.299683 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 199/412] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:40.306787 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c064b3b-4dd9-402a-899b-17d19c39cc5b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:40.441372 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b66a85-642a-4d90-ae55-2686fc559d4d None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.441555 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b66a85-642a-4d90-ae55-2686fc559d4d None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.441840 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b66a85-642a-4d90-ae55-2686fc559d4d None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.443306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 200/413] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:19:40.449114 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cb29e-5245-427c-a1cf-6de2b8084b30 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.449658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cb29e-5245-427c-a1cf-6de2b8084b30 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.450397 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3cb29e-5245-427c-a1cf-6de2b8084b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.461795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c064b3b-4dd9-402a-899b-17d19c39cc5b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:40.490389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-ce3cb29e-5245-427c-a1cf-6de2b8084b30 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:40.491596 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-ce3cb29e-5245-427c-a1cf-6de2b8084b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:40.510715 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b9988-0cb9-4a26-9f4c-e47730192eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:40.511380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b9988-0cb9-4a26-9f4c-e47730192eea None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.511690 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b9988-0cb9-4a26-9f4c-e47730192eea None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.512810 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b9988-0cb9-4a26-9f4c-e47730192eea None admin] PATH_INFO: `/v3/groups/875fe1a6a833450ebf5265f9ffba0ea8` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.514179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b9988-0cb9-4a26-9f4c-e47730192eea None admin] RBAC: Authorizing `identity:get_group(group_id=875fe1a6a833450ebf5265f9ffba0ea8)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:40.515989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b9988-0cb9-4a26-9f4c-e47730192eea None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:40.517735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 214/414] 10.222.0.58 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:40 2022] GET /identity/v3/groups/875fe1a6a833450ebf5265f9ffba0ea8 => generated 229 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:40.524791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b9988-0cb9-4a26-9f4c-e47730192eea None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:40.585207 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:40.603535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 201/415] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:40.615921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:40.654723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b9988-0cb9-4a26-9f4c-e47730192eea None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:40.689815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75211495-c9ab-48cf-a9fa-59c82e7440c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:40.690443 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75211495-c9ab-48cf-a9fa-59c82e7440c3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.690719 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75211495-c9ab-48cf-a9fa-59c82e7440c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.690990 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75211495-c9ab-48cf-a9fa-59c82e7440c3 None admin] PATH_INFO: `/v3/projects/ae0194ad9d2f4f1ebdd6b4e39c18be2c` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.692182 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75211495-c9ab-48cf-a9fa-59c82e7440c3 None admin] RBAC: Authorizing `identity:get_project(project_id=ae0194ad9d2f4f1ebdd6b4e39c18be2c)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:40.693560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75211495-c9ab-48cf-a9fa-59c82e7440c3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:40.700000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 215/416] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:40 2022] GET /identity/v3/projects/ae0194ad9d2f4f1ebdd6b4e39c18be2c => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:40.701842 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cb29e-5245-427c-a1cf-6de2b8084b30 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:40.713939 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac881edc-dc62-4509-8d83-abb377e3b241 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.714236 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac881edc-dc62-4509-8d83-abb377e3b241 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.714509 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac881edc-dc62-4509-8d83-abb377e3b241 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.716113 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 216/417] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:19:40.720961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d773fb6-5267-4bb5-9fb3-e353b2e5342e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.721238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d773fb6-5267-4bb5-9fb3-e353b2e5342e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.721507 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d773fb6-5267-4bb5-9fb3-e353b2e5342e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.741719 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3d773fb6-5267-4bb5-9fb3-e353b2e5342e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:40.742908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3d773fb6-5267-4bb5-9fb3-e353b2e5342e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:40.820742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 217/418] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:40.900159 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3cb29e-5245-427c-a1cf-6de2b8084b30 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:40.952403 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969db5fa-564e-46fe-8de8-6c061ff17245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:40.953007 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969db5fa-564e-46fe-8de8-6c061ff17245 None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:40.953295 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969db5fa-564e-46fe-8de8-6c061ff17245 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:40.953575 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969db5fa-564e-46fe-8de8-6c061ff17245 None admin] PATH_INFO: `/v3/projects/ae0194ad9d2f4f1ebdd6b4e39c18be2c/groups/875fe1a6a833450ebf5265f9ffba0ea8/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:40.955521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969db5fa-564e-46fe-8de8-6c061ff17245 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae0194ad9d2f4f1ebdd6b4e39c18be2c, group_id=875fe1a6a833450ebf5265f9ffba0ea8, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:40.956993 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969db5fa-564e-46fe-8de8-6c061ff17245 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:40.966804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d773fb6-5267-4bb5-9fb3-e353b2e5342e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:40.996598 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 202/419] 10.222.0.58 () {58 vars in 1442 bytes} [Wed Aug 24 12:19:40 2022] PUT /identity/v3/projects/ae0194ad9d2f4f1ebdd6b4e39c18be2c/groups/875fe1a6a833450ebf5265f9ffba0ea8/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:19:41.072019 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:41.146820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d773fb6-5267-4bb5-9fb3-e353b2e5342e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:41.168730 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969db5fa-564e-46fe-8de8-6c061ff17245 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:41.193464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29a868-25a3-4fe1-a785-8e279c8416ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:41.194562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29a868-25a3-4fe1-a785-8e279c8416ff None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.195076 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29a868-25a3-4fe1-a785-8e279c8416ff None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.195583 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29a868-25a3-4fe1-a785-8e279c8416ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.214365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29a868-25a3-4fe1-a785-8e279c8416ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:41.219250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-ec29a868-25a3-4fe1-a785-8e279c8416ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:41.219250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:41.219250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:41.219250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:41.219250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:41.219250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:41.219250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:41.219250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:41.219250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:41.219250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:41.219250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:41.219250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:41.219250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:41.219250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:41.219250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:41.219250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:41.220260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:41.220260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:41.220260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:41.220260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:41.220260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:41.220260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:41.220260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:41.220260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:41.220260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:41.220260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:41.220260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:41.220260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:41.220260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:41.220260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:41.220260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:41.220260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:41.222648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:41.222648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:41.222648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:41.222648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:41.222648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:41.222648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:41.222648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:41.222648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:41.222648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:41.222648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:41.222648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:41.222648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:41.222648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 218/420] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:41.226368 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29a868-25a3-4fe1-a785-8e279c8416ff None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:41.239687 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:41.297890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969db5fa-564e-46fe-8de8-6c061ff17245 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:41.334717 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fa235-eb97-4d72-a230-116ac418f8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:41.335417 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fa235-eb97-4d72-a230-116ac418f8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.335688 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fa235-eb97-4d72-a230-116ac418f8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.336010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fa235-eb97-4d72-a230-116ac418f8d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.337593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29a868-25a3-4fe1-a785-8e279c8416ff None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:41.346514 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7fa235-eb97-4d72-a230-116ac418f8d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:41.351376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-0a7fa235-eb97-4d72-a230-116ac418f8d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:41.351376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:41.351376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:41.351376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:41.351376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:41.351376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:41.351376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:41.351376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:41.351376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:41.351376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:41.351376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:41.351376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:41.351376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:41.351376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:41.351376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:41.351376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:41.353470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:41.353470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:41.353470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:41.353470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:41.353470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:41.353470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:41.353470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:41.353470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:41.353470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:41.353470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:41.353470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:41.353470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:41.353470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:41.353470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:41.353470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:41.353470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:41.355992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:41.355992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:41.355992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:41.355992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:41.355992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:41.355992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:41.355992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:41.355992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:41.355992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:41.355992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:41.355992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:41.355992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:41.355992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 203/421] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:41.362428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:41.367375 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fa235-eb97-4d72-a230-116ac418f8d8 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:41.375689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd7f9b-0f97-4561-b3a7-3d7402028eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:41.376306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd7f9b-0f97-4561-b3a7-3d7402028eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.376590 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd7f9b-0f97-4561-b3a7-3d7402028eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.376867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd7f9b-0f97-4561-b3a7-3d7402028eb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.377537 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd7f9b-0f97-4561-b3a7-3d7402028eb6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:41.377845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd7f9b-0f97-4561-b3a7-3d7402028eb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:41.379246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd7f9b-0f97-4561-b3a7-3d7402028eb6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:41.388357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 219/422] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:41.393817 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd7f9b-0f97-4561-b3a7-3d7402028eb6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:41.500572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd7f9b-0f97-4561-b3a7-3d7402028eb6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:41.538530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59e76c-27f3-46b8-9e2a-296620f2cbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:41.539057 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7fa235-eb97-4d72-a230-116ac418f8d8 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:41.539567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59e76c-27f3-46b8-9e2a-296620f2cbe6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.539567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59e76c-27f3-46b8-9e2a-296620f2cbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.539900 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac59e76c-27f3-46b8-9e2a-296620f2cbe6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.562585 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:41.613276 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41292d8-d0cb-46c3-a948-5bcb1cd0ff2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:41.613894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41292d8-d0cb-46c3-a948-5bcb1cd0ff2b None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.614175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41292d8-d0cb-46c3-a948-5bcb1cd0ff2b None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.614447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41292d8-d0cb-46c3-a948-5bcb1cd0ff2b None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.615117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41292d8-d0cb-46c3-a948-5bcb1cd0ff2b None admin] RBAC: Adding query filter params (name=member) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:41.615420 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41292d8-d0cb-46c3-a948-5bcb1cd0ff2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:41.616912 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41292d8-d0cb-46c3-a948-5bcb1cd0ff2b None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:41.631654 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 204/423] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:41.632121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:41.638483 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41292d8-d0cb-46c3-a948-5bcb1cd0ff2b None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:41.648820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59e76c-27f3-46b8-9e2a-296620f2cbe6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:41.650613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac59e76c-27f3-46b8-9e2a-296620f2cbe6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:41.800705 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 220/424] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:41.806841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59e76c-27f3-46b8-9e2a-296620f2cbe6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:41.833555 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41292d8-d0cb-46c3-a948-5bcb1cd0ff2b None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:41.906248 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:41.926318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac59e76c-27f3-46b8-9e2a-296620f2cbe6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:41.930922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c965d95-74d3-4b05-b107-8f1edafb225c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:41.931544 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c965d95-74d3-4b05-b107-8f1edafb225c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.931851 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c965d95-74d3-4b05-b107-8f1edafb225c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.932127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c965d95-74d3-4b05-b107-8f1edafb225c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.968255 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6626571-2681-42d8-a702-83cb67cfb273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:41.969439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6626571-2681-42d8-a702-83cb67cfb273 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:41.969912 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6626571-2681-42d8-a702-83cb67cfb273 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:41.970372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6626571-2681-42d8-a702-83cb67cfb273 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:41.971753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6626571-2681-42d8-a702-83cb67cfb273 None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:41.974008 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6626571-2681-42d8-a702-83cb67cfb273 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:41.976179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 221/425] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:41 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:41.982066 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6626571-2681-42d8-a702-83cb67cfb273 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.011928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:42.065358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c965d95-74d3-4b05-b107-8f1edafb225c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:42.066740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c965d95-74d3-4b05-b107-8f1edafb225c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:42.171275 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6626571-2681-42d8-a702-83cb67cfb273 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:42.216380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6d3e4-0b1b-433d-9c93-2c5a971c67a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:42.217328 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6d3e4-0b1b-433d-9c93-2c5a971c67a3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.217611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6d3e4-0b1b-433d-9c93-2c5a971c67a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.217881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6d3e4-0b1b-433d-9c93-2c5a971c67a3 None admin] PATH_INFO: `/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.219438 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d6d3e4-0b1b-433d-9c93-2c5a971c67a3 None admin] RBAC: Authorizing `identity:get_group(group_id=0c59fa3bb1ec4933aec15e5f32612ef7)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:42.221335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d6d3e4-0b1b-433d-9c93-2c5a971c67a3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:42.223792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 222/426] 10.222.0.58 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:41 2022] GET /identity/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7 => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:42.224965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab02416-6389-4cc6-a006-b1ed2588b880 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.225245 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab02416-6389-4cc6-a006-b1ed2588b880 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.225363 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 205/427] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:19:42.225567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab02416-6389-4cc6-a006-b1ed2588b880 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.226582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2bbc7-23ef-4dc7-87ea-6f6537723e45 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.226871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2bbc7-23ef-4dc7-87ea-6f6537723e45 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.227136 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 223/428] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:42.230862 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2bbc7-23ef-4dc7-87ea-6f6537723e45 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.232283 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 206/429] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:42.232722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab02416-6389-4cc6-a006-b1ed2588b880 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.233011 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2bbc7-23ef-4dc7-87ea-6f6537723e45 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.258587 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:42.310098 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:42.345188 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab02416-6389-4cc6-a006-b1ed2588b880 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:42.347749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2bbc7-23ef-4dc7-87ea-6f6537723e45 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:42.383192 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358431e-d6a1-4aa2-93f0-05a621850609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:42.383365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c001a46-710a-4898-af4b-6bfeb9a28ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:42.383941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c001a46-710a-4898-af4b-6bfeb9a28ffe None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.384226 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c001a46-710a-4898-af4b-6bfeb9a28ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.384504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c001a46-710a-4898-af4b-6bfeb9a28ffe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.384924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358431e-d6a1-4aa2-93f0-05a621850609 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.385233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c001a46-710a-4898-af4b-6bfeb9a28ffe None admin] RBAC: Adding query filter params (group.id=0c59fa3bb1ec4933aec15e5f32612ef7, role.id=4e002a22056744acb8c40a3236cab404, scope.project.id=c480d76c537348a7a86ae331f79d5e39) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:42.385546 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c001a46-710a-4898-af4b-6bfeb9a28ffe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:42.385918 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358431e-d6a1-4aa2-93f0-05a621850609 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.386422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358431e-d6a1-4aa2-93f0-05a621850609 None admin] PATH_INFO: `/v3/projects/c480d76c537348a7a86ae331f79d5e39` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.387017 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c001a46-710a-4898-af4b-6bfeb9a28ffe None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:42.387995 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358431e-d6a1-4aa2-93f0-05a621850609 None admin] RBAC: Authorizing `identity:get_project(project_id=c480d76c537348a7a86ae331f79d5e39)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:42.389623 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358431e-d6a1-4aa2-93f0-05a621850609 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:42.391837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 224/430] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:42 2022] GET /identity/v3/projects/c480d76c537348a7a86ae331f79d5e39 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:42.402077 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 207/431] 10.222.0.58 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:42 2022] GET /identity/v3/role_assignments?group.id=0c59fa3bb1ec4933aec15e5f32612ef7&scope.project.id=c480d76c537348a7a86ae331f79d5e39&role.id=4e002a22056744acb8c40a3236cab404 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:42.402077 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa540aac-f46d-4292-a5b4-953e871806ca None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.402077 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa540aac-f46d-4292-a5b4-953e871806ca None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.402077 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa540aac-f46d-4292-a5b4-953e871806ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.402514 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb467785-40c3-4a07-a977-f9f48ddc452e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.402514 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb467785-40c3-4a07-a977-f9f48ddc452e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.402514 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb467785-40c3-4a07-a977-f9f48ddc452e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.422294 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-cb467785-40c3-4a07-a977-f9f48ddc452e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:42.422505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-fa540aac-f46d-4292-a5b4-953e871806ca None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:42.423513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fa540aac-f46d-4292-a5b4-953e871806ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:42.423646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cb467785-40c3-4a07-a977-f9f48ddc452e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:42.504012 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 208/432] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:42.504983 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa540aac-f46d-4292-a5b4-953e871806ca None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.512984 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 225/433] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:42.576054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:42.615246 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa540aac-f46d-4292-a5b4-953e871806ca None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:42.654777 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee988fd-aa9c-4608-b9b6-46e55efffbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:42.655427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee988fd-aa9c-4608-b9b6-46e55efffbd1 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.655734 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee988fd-aa9c-4608-b9b6-46e55efffbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.656010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee988fd-aa9c-4608-b9b6-46e55efffbd1 None admin] PATH_INFO: `/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.657182 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee988fd-aa9c-4608-b9b6-46e55efffbd1 None admin] RBAC: Authorizing `identity:get_group(group_id=0c59fa3bb1ec4933aec15e5f32612ef7)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:42.658686 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee988fd-aa9c-4608-b9b6-46e55efffbd1 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:42.660652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 209/434] 10.222.0.58 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:42 2022] GET /identity/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:42.666296 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb467785-40c3-4a07-a977-f9f48ddc452e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.692597 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:42.794875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb467785-40c3-4a07-a977-f9f48ddc452e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:42.833988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6316adb-ac5e-4de6-8604-4604b32c78f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:42.834946 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6316adb-ac5e-4de6-8604-4604b32c78f3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:42.835447 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6316adb-ac5e-4de6-8604-4604b32c78f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:42.835956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6316adb-ac5e-4de6-8604-4604b32c78f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:42.836911 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6316adb-ac5e-4de6-8604-4604b32c78f3 None admin] RBAC: Adding query filter params (group.id=0c59fa3bb1ec4933aec15e5f32612ef7, role.id=156b5bb1275a4dbeb61dbdade2c20d21, scope.project.id=c480d76c537348a7a86ae331f79d5e39) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:42.837438 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6316adb-ac5e-4de6-8604-4604b32c78f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:42.839656 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6316adb-ac5e-4de6-8604-4604b32c78f3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:42.843666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee988fd-aa9c-4608-b9b6-46e55efffbd1 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.846635 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 226/435] 10.222.0.58 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:42 2022] GET /identity/v3/role_assignments?group.id=0c59fa3bb1ec4933aec15e5f32612ef7&scope.project.id=c480d76c537348a7a86ae331f79d5e39&role.id=156b5bb1275a4dbeb61dbdade2c20d21 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:42.847446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6316adb-ac5e-4de6-8604-4604b32c78f3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:42.958023 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:42.975156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee988fd-aa9c-4608-b9b6-46e55efffbd1 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:42.980476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6316adb-ac5e-4de6-8604-4604b32c78f3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:43.016870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:43.026763 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f9ea5-81b5-42ba-89c4-a2066c46d3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.027042 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7b72c-00f8-4ead-a551-6e97340a7d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.027408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f9ea5-81b5-42ba-89c4-a2066c46d3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.027691 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7b72c-00f8-4ead-a551-6e97340a7d11 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.027803 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f9ea5-81b5-42ba-89c4-a2066c46d3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.027973 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7b72c-00f8-4ead-a551-6e97340a7d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.028065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f9ea5-81b5-42ba-89c4-a2066c46d3f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.028219 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7b72c-00f8-4ead-a551-6e97340a7d11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.039850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f9ea5-81b5-42ba-89c4-a2066c46d3f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:43.041146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7b72c-00f8-4ead-a551-6e97340a7d11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:43.045872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-1c0f9ea5-81b5-42ba-89c4-a2066c46d3f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:43.045872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:43.045872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:43.045872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:43.045872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:43.045872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:43.045872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:43.045872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:43.045872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:43.045872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:43.045872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:43.045872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:43.045872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:43.045872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:43.045872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:43.045872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:43.046753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:43.046753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:43.046753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:43.046753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:43.046753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:43.046753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:43.046753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:43.046753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:43.046753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:43.046753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:43.046753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:43.046753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:43.046753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:43.046753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:43.046753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:43.046753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:43.047592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:43.047592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:43.047592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:43.047592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:43.047592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:43.047592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:43.047592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:43.047592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:43.047592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:43.047592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:43.047592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:43.047592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:43.047592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 227/436] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:43.048287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-a1f7b72c-00f8-4ead-a551-6e97340a7d11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:43.048287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:43.048287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:43.048287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:43.048287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:43.048287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:43.048287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:43.048287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:43.048287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:43.048287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:43.048287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:43.048287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:43.048287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:43.048287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:43.048287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:43.048287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:43.049089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:43.049089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:43.049089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:43.049089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:43.049089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:43.049089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:43.049089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:43.049089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:43.049089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:43.049089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:43.049089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:43.049089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:43.049089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:43.049089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:43.049089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:43.049089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:43.049890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:43.049890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:43.049890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:43.049890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:43.049890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:43.049890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:43.049890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:43.049890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:43.049890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:43.049890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:43.049890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 12:19:43.049890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:43.049890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 210/437] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:43.054053 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f9ea5-81b5-42ba-89c4-a2066c46d3f3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:43.056447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7b72c-00f8-4ead-a551-6e97340a7d11 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:43.197046 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7b72c-00f8-4ead-a551-6e97340a7d11 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:43.199226 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f9ea5-81b5-42ba-89c4-a2066c46d3f3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:43.242117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896a9fe-c1f1-4064-bf5f-b610f4931721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.242738 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896a9fe-c1f1-4064-bf5f-b610f4931721 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.243018 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896a9fe-c1f1-4064-bf5f-b610f4931721 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.243291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896a9fe-c1f1-4064-bf5f-b610f4931721 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.244013 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896a9fe-c1f1-4064-bf5f-b610f4931721 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:43.244318 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896a9fe-c1f1-4064-bf5f-b610f4931721 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:43.246519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896a9fe-c1f1-4064-bf5f-b610f4931721 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:43.252973 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922fc14-44a0-46ad-9352-3298a1b53d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.253909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922fc14-44a0-46ad-9352-3298a1b53d89 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.254390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922fc14-44a0-46ad-9352-3298a1b53d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.254874 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922fc14-44a0-46ad-9352-3298a1b53d89 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.255790 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922fc14-44a0-46ad-9352-3298a1b53d89 None admin] RBAC: Adding query filter params (name=member) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:43.256148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 211/438] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:43.256512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922fc14-44a0-46ad-9352-3298a1b53d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:43.258062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922fc14-44a0-46ad-9352-3298a1b53d89 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:43.262114 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896a9fe-c1f1-4064-bf5f-b610f4931721 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:43.271895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 228/439] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:19:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:43.278825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922fc14-44a0-46ad-9352-3298a1b53d89 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:43.283800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:43.327638 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:43.383134 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896a9fe-c1f1-4064-bf5f-b610f4931721 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:43.409271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922fc14-44a0-46ad-9352-3298a1b53d89 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:43.424120 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930437a8-0a8a-4958-be2f-7201862e49b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.424793 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930437a8-0a8a-4958-be2f-7201862e49b3 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.425074 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930437a8-0a8a-4958-be2f-7201862e49b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.425335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930437a8-0a8a-4958-be2f-7201862e49b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.452881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be4063-ef7a-49f4-aa33-27e6534ba536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.453420 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be4063-ef7a-49f4-aa33-27e6534ba536 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.453704 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be4063-ef7a-49f4-aa33-27e6534ba536 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.454247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be4063-ef7a-49f4-aa33-27e6534ba536 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.510290 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930437a8-0a8a-4958-be2f-7201862e49b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:43.512849 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930437a8-0a8a-4958-be2f-7201862e49b3 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:43.547200 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be4063-ef7a-49f4-aa33-27e6534ba536 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:43.547851 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be4063-ef7a-49f4-aa33-27e6534ba536 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:43.550596 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:43.571083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:43.617821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 212/440] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:19:43.619125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef1423-de0c-41da-9011-53251af4ab9a None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.619414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef1423-de0c-41da-9011-53251af4ab9a None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.619730 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef1423-de0c-41da-9011-53251af4ab9a None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.621194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 213/441] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:43.626552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b075a8-e58a-497e-a364-e649d8307ef1 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.626841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b075a8-e58a-497e-a364-e649d8307ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.627106 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b075a8-e58a-497e-a364-e649d8307ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.645371 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-89b075a8-e58a-497e-a364-e649d8307ef1 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:43.646550 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-89b075a8-e58a-497e-a364-e649d8307ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:43.648550 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 229/442] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:19:43.649381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be4063-ef7a-49f4-aa33-27e6534ba536 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:43.719928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 214/443] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:43.720819 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b075a8-e58a-497e-a364-e649d8307ef1 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:43.747845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be4063-ef7a-49f4-aa33-27e6534ba536 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:43.750430 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:43.786442 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa6a15-d995-47c8-86c7-fe7c6a738047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.787062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa6a15-d995-47c8-86c7-fe7c6a738047 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.787338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa6a15-d995-47c8-86c7-fe7c6a738047 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.787606 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa6a15-d995-47c8-86c7-fe7c6a738047 None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.788725 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa6a15-d995-47c8-86c7-fe7c6a738047 None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:43.790163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa6a15-d995-47c8-86c7-fe7c6a738047 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:43.791785 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 230/444] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:43 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:43.797539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa6a15-d995-47c8-86c7-fe7c6a738047 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:43.816929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:43.837003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b075a8-e58a-497e-a364-e649d8307ef1 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:43.872719 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e584c-129d-4bf6-8109-71f1465e4eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.873345 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e584c-129d-4bf6-8109-71f1465e4eeb None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.873620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e584c-129d-4bf6-8109-71f1465e4eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.876652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e584c-129d-4bf6-8109-71f1465e4eeb None admin] PATH_INFO: `/v3/projects/d9b6194f22b34cd695de9f04dace7e99` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.879312 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e584c-129d-4bf6-8109-71f1465e4eeb None admin] RBAC: Authorizing `identity:get_project(project_id=d9b6194f22b34cd695de9f04dace7e99)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:43.880910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e584c-129d-4bf6-8109-71f1465e4eeb None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:43.882721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 215/445] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:43 2022] GET /identity/v3/projects/d9b6194f22b34cd695de9f04dace7e99 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:43.888874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e584c-129d-4bf6-8109-71f1465e4eeb None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:43.921029 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa6a15-d995-47c8-86c7-fe7c6a738047 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:43.958718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8ad6d-bd62-486c-8460-ca1f8d510520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:43.959339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8ad6d-bd62-486c-8460-ca1f8d510520 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:43.959607 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8ad6d-bd62-486c-8460-ca1f8d510520 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:43.959901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8ad6d-bd62-486c-8460-ca1f8d510520 None admin] PATH_INFO: `/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:43.960997 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8ad6d-bd62-486c-8460-ca1f8d510520 None admin] RBAC: Authorizing `identity:get_group(group_id=0c59fa3bb1ec4933aec15e5f32612ef7)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:43.962440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8ad6d-bd62-486c-8460-ca1f8d510520 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:43.968933 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 231/446] 10.222.0.58 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:43 2022] GET /identity/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:43.970156 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8ad6d-bd62-486c-8460-ca1f8d510520 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:43.997917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e584c-129d-4bf6-8109-71f1465e4eeb None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:44.013615 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:44.032561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9e3d1-c464-4b41-bcfc-5ae0527e3356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:44.033457 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9e3d1-c464-4b41-bcfc-5ae0527e3356 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:44.033733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9e3d1-c464-4b41-bcfc-5ae0527e3356 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:44.034003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9e3d1-c464-4b41-bcfc-5ae0527e3356 None admin] PATH_INFO: `/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:44.035101 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9e3d1-c464-4b41-bcfc-5ae0527e3356 None admin] RBAC: Authorizing `identity:get_group(group_id=0c59fa3bb1ec4933aec15e5f32612ef7)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:44.036570 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9e3d1-c464-4b41-bcfc-5ae0527e3356 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:44.038133 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 216/447] 10.222.0.58 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:43 2022] GET /identity/v3/groups/0c59fa3bb1ec4933aec15e5f32612ef7 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:44.040546 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9e3d1-c464-4b41-bcfc-5ae0527e3356 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:44.062443 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:44.072539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8ad6d-bd62-486c-8460-ca1f8d510520 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:44.106525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4b6a7-365e-46fd-8f54-06dcc463b6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:44.107123 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4b6a7-365e-46fd-8f54-06dcc463b6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:44.107398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4b6a7-365e-46fd-8f54-06dcc463b6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:44.107683 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4b6a7-365e-46fd-8f54-06dcc463b6a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:44.108410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4b6a7-365e-46fd-8f54-06dcc463b6a3 None admin] RBAC: Adding query filter params (group.id=0c59fa3bb1ec4933aec15e5f32612ef7, role.id=4e002a22056744acb8c40a3236cab404, scope.project.id=d9b6194f22b34cd695de9f04dace7e99) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:44.108712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4b6a7-365e-46fd-8f54-06dcc463b6a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:44.110141 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4b6a7-365e-46fd-8f54-06dcc463b6a3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:44.115929 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 232/448] 10.222.0.58 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:43 2022] GET /identity/v3/role_assignments?group.id=0c59fa3bb1ec4933aec15e5f32612ef7&scope.project.id=d9b6194f22b34cd695de9f04dace7e99&role.id=4e002a22056744acb8c40a3236cab404 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:44.116539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4b6a7-365e-46fd-8f54-06dcc463b6a3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:44.165953 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9e3d1-c464-4b41-bcfc-5ae0527e3356 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:44.204876 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca9330-2d0f-4eab-a4e4-76dec27e5df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:44.205526 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca9330-2d0f-4eab-a4e4-76dec27e5df4 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:44.205807 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca9330-2d0f-4eab-a4e4-76dec27e5df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:44.206075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca9330-2d0f-4eab-a4e4-76dec27e5df4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:44.219905 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca9330-2d0f-4eab-a4e4-76dec27e5df4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:44.225511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-f9ca9330-2d0f-4eab-a4e4-76dec27e5df4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:44.225511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:44.225511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:44.225511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:44.225511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:44.225511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:44.225511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:44.225511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:44.225511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:44.225511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:44.225511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:44.225511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:44.225511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:44.225511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:44.225511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:44.225511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:44.226454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:44.226454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:44.226454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:44.226454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:44.226454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:44.226454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:44.226454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:44.226454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:44.226454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:44.226454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:44.226454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:44.226454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:44.226454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:44.226454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:44.226454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:44.226454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:44.227266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:44.227266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:44.227266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:44.227266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:44.227266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:44.227266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:44.227266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:44.227266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:44.227266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:44.227266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:44.227266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:19:44.227266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:44.227266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 217/449] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:44.233213 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca9330-2d0f-4eab-a4e4-76dec27e5df4 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:44.238893 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4b6a7-365e-46fd-8f54-06dcc463b6a3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:44.241475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:44.278819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf32440-4dc8-4a47-a3f8-d3650f7c071b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:44.279429 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf32440-4dc8-4a47-a3f8-d3650f7c071b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:44.280367 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf32440-4dc8-4a47-a3f8-d3650f7c071b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:44.280367 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf32440-4dc8-4a47-a3f8-d3650f7c071b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:44.280806 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf32440-4dc8-4a47-a3f8-d3650f7c071b None admin] RBAC: Adding query filter params (group.id=0c59fa3bb1ec4933aec15e5f32612ef7, role.id=156b5bb1275a4dbeb61dbdade2c20d21, scope.project.id=d9b6194f22b34cd695de9f04dace7e99) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:44.281116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf32440-4dc8-4a47-a3f8-d3650f7c071b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:44.283457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf32440-4dc8-4a47-a3f8-d3650f7c071b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:44.288868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 233/450] 10.222.0.58 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:44 2022] GET /identity/v3/role_assignments?group.id=0c59fa3bb1ec4933aec15e5f32612ef7&scope.project.id=d9b6194f22b34cd695de9f04dace7e99&role.id=156b5bb1275a4dbeb61dbdade2c20d21 => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:44.336356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:44.357810 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca9330-2d0f-4eab-a4e4-76dec27e5df4 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:44.393649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2944c-3f9c-459a-8b90-4d86855fff38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:44.394272 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2944c-3f9c-459a-8b90-4d86855fff38 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:44.394554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2944c-3f9c-459a-8b90-4d86855fff38 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:44.394828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2944c-3f9c-459a-8b90-4d86855fff38 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:44.395511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2944c-3f9c-459a-8b90-4d86855fff38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:44.395848 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2944c-3f9c-459a-8b90-4d86855fff38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:44.397280 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2944c-3f9c-459a-8b90-4d86855fff38 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:44.406565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 218/451] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:44.412258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf32440-4dc8-4a47-a3f8-d3650f7c071b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:44.516303 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf32440-4dc8-4a47-a3f8-d3650f7c071b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:44.550353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fffcda-3334-4791-8d51-22389817f62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:44.550964 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fffcda-3334-4791-8d51-22389817f62a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:44.551247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fffcda-3334-4791-8d51-22389817f62a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:44.551529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fffcda-3334-4791-8d51-22389817f62a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:44.586885 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:44.620768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fffcda-3334-4791-8d51-22389817f62a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:44.622238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fffcda-3334-4791-8d51-22389817f62a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:44.721705 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 234/452] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 12:19:44.728124 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2944c-3f9c-459a-8b90-4d86855fff38 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:44.829096 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2944c-3f9c-459a-8b90-4d86855fff38 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:44.861030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86690225-64d8-4848-b1cb-25fce02182f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:44.861649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86690225-64d8-4848-b1cb-25fce02182f7 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:44.861931 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86690225-64d8-4848-b1cb-25fce02182f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:44.862204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86690225-64d8-4848-b1cb-25fce02182f7 None admin] PATH_INFO: `/v3/projects/ae0194ad9d2f4f1ebdd6b4e39c18be2c` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:44.863307 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86690225-64d8-4848-b1cb-25fce02182f7 None admin] RBAC: Authorizing `identity:get_project(project_id=ae0194ad9d2f4f1ebdd6b4e39c18be2c)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:44.864781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86690225-64d8-4848-b1cb-25fce02182f7 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:44.866405 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 219/453] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:19:44 2022] GET /identity/v3/projects/ae0194ad9d2f4f1ebdd6b4e39c18be2c => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:44.872672 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fffcda-3334-4791-8d51-22389817f62a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:44.937349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:44.969727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fffcda-3334-4791-8d51-22389817f62a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:45.002136 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2cf838-16aa-4f46-b2c4-d288d92b6fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:45.002750 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cf838-16aa-4f46-b2c4-d288d92b6fee None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:45.003027 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cf838-16aa-4f46-b2c4-d288d92b6fee None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:45.003298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2cf838-16aa-4f46-b2c4-d288d92b6fee None admin] PATH_INFO: `/v3/groups/875fe1a6a833450ebf5265f9ffba0ea8` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:45.004410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cf838-16aa-4f46-b2c4-d288d92b6fee None admin] RBAC: Authorizing `identity:get_group(group_id=875fe1a6a833450ebf5265f9ffba0ea8)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:45.005839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2cf838-16aa-4f46-b2c4-d288d92b6fee None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:45.007398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 235/454] 10.222.0.58 () {56 vars in 1180 bytes} [Wed Aug 24 12:19:44 2022] GET /identity/v3/groups/875fe1a6a833450ebf5265f9ffba0ea8 => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:45.014143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86690225-64d8-4848-b1cb-25fce02182f7 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:45.026731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:45.113149 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86690225-64d8-4848-b1cb-25fce02182f7 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:45.145715 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e673d7-f801-4ba2-a026-42f01cd68dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:45.146308 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e673d7-f801-4ba2-a026-42f01cd68dfc None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:45.146590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e673d7-f801-4ba2-a026-42f01cd68dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:45.146866 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e673d7-f801-4ba2-a026-42f01cd68dfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:45.147587 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e673d7-f801-4ba2-a026-42f01cd68dfc None admin] RBAC: Adding query filter params (group.id=875fe1a6a833450ebf5265f9ffba0ea8, role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=ae0194ad9d2f4f1ebdd6b4e39c18be2c) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:45.147931 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e673d7-f801-4ba2-a026-42f01cd68dfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:45.149373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e673d7-f801-4ba2-a026-42f01cd68dfc None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:45.155090 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 220/455] 10.222.0.58 () {56 vars in 1376 bytes} [Wed Aug 24 12:19:45 2022] GET /identity/v3/role_assignments?group.id=875fe1a6a833450ebf5265f9ffba0ea8&scope.project.id=ae0194ad9d2f4f1ebdd6b4e39c18be2c&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:47.570732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57693ea-a142-47c0-ac83-c6e6ddd26f0f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:47.571258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57693ea-a142-47c0-ac83-c6e6ddd26f0f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:47.571258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57693ea-a142-47c0-ac83-c6e6ddd26f0f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:47.572915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 236/456] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:47.578369 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec125a-5163-404d-a2c4-93e7009e5c6f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:47.578814 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec125a-5163-404d-a2c4-93e7009e5c6f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:47.578986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec125a-5163-404d-a2c4-93e7009e5c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:47.601374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-aaec125a-5163-404d-a2c4-93e7009e5c6f None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:47.602565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-aaec125a-5163-404d-a2c4-93e7009e5c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:47.681519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 221/457] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:47.718590 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7944ab-d063-4c0c-9eeb-42f5f643ea3b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:47.718908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7944ab-d063-4c0c-9eeb-42f5f643ea3b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:47.719143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7944ab-d063-4c0c-9eeb-42f5f643ea3b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:47.720525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 237/458] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:47.725791 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b2ad3-3ac6-4336-885e-83b78a8c2727 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:47.726077 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b2ad3-3ac6-4336-885e-83b78a8c2727 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:47.726349 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b2ad3-3ac6-4336-885e-83b78a8c2727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:47.734319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:47.745914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-c38b2ad3-3ac6-4336-885e-83b78a8c2727 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:47.747562 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c38b2ad3-3ac6-4336-885e-83b78a8c2727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:47.823537 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 222/459] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:48.087996 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7944ab-d063-4c0c-9eeb-42f5f643ea3b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:48.185681 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5d9b5-93c1-4b13-b1f0-0aa58c30dc2c None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.186314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5d9b5-93c1-4b13-b1f0-0aa58c30dc2c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.186809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5d9b5-93c1-4b13-b1f0-0aa58c30dc2c None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.189694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 223/460] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:19:48.194084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4230b-1a0e-4dc3-a3a1-bdd09fad048f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.194446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4230b-1a0e-4dc3-a3a1-bdd09fad048f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.194797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4230b-1a0e-4dc3-a3a1-bdd09fad048f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.201384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7944ab-d063-4c0c-9eeb-42f5f643ea3b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:48.216366 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-27e4230b-1a0e-4dc3-a3a1-bdd09fad048f None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:48.217676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-27e4230b-1a0e-4dc3-a3a1-bdd09fad048f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:48.239813 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e78d6-7ec8-49a8-919b-a56d6138d2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:48.240846 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e78d6-7ec8-49a8-919b-a56d6138d2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.241330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e78d6-7ec8-49a8-919b-a56d6138d2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.241782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e78d6-7ec8-49a8-919b-a56d6138d2e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.244873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:48.254796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e78d6-7ec8-49a8-919b-a56d6138d2e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:48.256524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e78d6-7ec8-49a8-919b-a56d6138d2e6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:48.273365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-289e78d6-7ec8-49a8-919b-a56d6138d2e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:48.273365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:48.273365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:48.273365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:48.273365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:48.273365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:48.273365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:48.273365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:48.273365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:48.273365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:48.273365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:48.273365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:48.273365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:48.273365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:48.273365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:48.273365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:48.274313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:48.275186 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:48.275186 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:48.275186 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:48.277232 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 238/461] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:48.278297 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e78d6-7ec8-49a8-919b-a56d6138d2e6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:48.294544 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 224/462] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:48.300185 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4230b-1a0e-4dc3-a3a1-bdd09fad048f None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:48.382416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:48.390885 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e78d6-7ec8-49a8-919b-a56d6138d2e6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:48.426034 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27875bda-0f82-47ec-a3fa-7f56cc80bbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:48.426716 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27875bda-0f82-47ec-a3fa-7f56cc80bbd4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.426999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27875bda-0f82-47ec-a3fa-7f56cc80bbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.427270 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27875bda-0f82-47ec-a3fa-7f56cc80bbd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.428310 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4230b-1a0e-4dc3-a3a1-bdd09fad048f None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:48.438348 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27875bda-0f82-47ec-a3fa-7f56cc80bbd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:48.439836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27875bda-0f82-47ec-a3fa-7f56cc80bbd4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:48.454831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-27875bda-0f82-47ec-a3fa-7f56cc80bbd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:48.454831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:48.454831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:48.454831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:48.454831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:48.454831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:48.454831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:48.454831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:48.454831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:48.454831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:48.454831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:48.454831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:48.454831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:48.454831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:48.454831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:48.454831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:48.455873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:48.456916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:48.456916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:48.456916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:48.460668 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 239/463] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:48.461925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd80f1-01bd-4ab1-a880-37a1e8217d9f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.462206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd80f1-01bd-4ab1-a880-37a1e8217d9f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.462476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd80f1-01bd-4ab1-a880-37a1e8217d9f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.464780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 240/464] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:48.465437 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd80f1-01bd-4ab1-a880-37a1e8217d9f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:48.469725 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db9ed1-df6f-43d4-b105-b5c99c420510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:48.470507 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db9ed1-df6f-43d4-b105-b5c99c420510 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.470946 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db9ed1-df6f-43d4-b105-b5c99c420510 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.471356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db9ed1-df6f-43d4-b105-b5c99c420510 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.472165 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db9ed1-df6f-43d4-b105-b5c99c420510 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:48.472720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db9ed1-df6f-43d4-b105-b5c99c420510 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:48.474345 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db9ed1-df6f-43d4-b105-b5c99c420510 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:48.487945 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 225/465] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:48.494008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40aa33-a338-44f0-9c02-b637957554fc None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.494451 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40aa33-a338-44f0-9c02-b637957554fc None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.494868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40aa33-a338-44f0-9c02-b637957554fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.514557 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-ed40aa33-a338-44f0-9c02-b637957554fc None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:48.515682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:48.517178 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-ed40aa33-a338-44f0-9c02-b637957554fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:48.578941 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd80f1-01bd-4ab1-a880-37a1e8217d9f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:48.596037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 226/466] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:48.596037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40aa33-a338-44f0-9c02-b637957554fc None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:48.622132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829296b1-be02-4995-8708-96652ef4ea41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:48.623006 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829296b1-be02-4995-8708-96652ef4ea41 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.623292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829296b1-be02-4995-8708-96652ef4ea41 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.623565 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829296b1-be02-4995-8708-96652ef4ea41 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.624246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829296b1-be02-4995-8708-96652ef4ea41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:48.624803 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829296b1-be02-4995-8708-96652ef4ea41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:48.626881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829296b1-be02-4995-8708-96652ef4ea41 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:48.647014 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 241/467] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:48.648078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829296b1-be02-4995-8708-96652ef4ea41 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:48.664333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:48.710173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40aa33-a338-44f0-9c02-b637957554fc None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:48.745537 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e8a39-8ce9-4496-92a3-982035f9f9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:48.746144 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e8a39-8ce9-4496-92a3-982035f9f9ba None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.746426 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e8a39-8ce9-4496-92a3-982035f9f9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.746702 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e8a39-8ce9-4496-92a3-982035f9f9ba None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.747340 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e8a39-8ce9-4496-92a3-982035f9f9ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:48.749429 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e8a39-8ce9-4496-92a3-982035f9f9ba None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:48.757471 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:48.763204 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829296b1-be02-4995-8708-96652ef4ea41 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:48.787675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 227/468] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:48 2022] POST /identity/v3/users => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:48.787675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e8a39-8ce9-4496-92a3-982035f9f9ba None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:48.834913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72299b-af69-4d31-ba9a-1e4ca21499ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:48.835570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72299b-af69-4d31-ba9a-1e4ca21499ab None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:48.835875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72299b-af69-4d31-ba9a-1e4ca21499ab None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:48.836148 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72299b-af69-4d31-ba9a-1e4ca21499ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:48.851212 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72299b-af69-4d31-ba9a-1e4ca21499ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:48.852892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d72299b-af69-4d31-ba9a-1e4ca21499ab None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:48.871961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-8d72299b-af69-4d31-ba9a-1e4ca21499ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:48.871961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:48.871961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:48.871961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:48.871961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:48.871961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:48.871961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:48.871961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:48.871961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:48.871961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:48.871961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:48.871961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:48.871961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:48.871961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:48.871961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:48.871961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:48.872943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:48.873792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:48.873792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:48.873792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:48.873792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 242/469] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:48.889174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72299b-af69-4d31-ba9a-1e4ca21499ab None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:48.954601 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e8a39-8ce9-4496-92a3-982035f9f9ba None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:48.967533 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:48.999896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff260b7-5c67-4854-9e94-08ae5aecacef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:49.000518 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff260b7-5c67-4854-9e94-08ae5aecacef None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:49.000800 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff260b7-5c67-4854-9e94-08ae5aecacef None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:49.001078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff260b7-5c67-4854-9e94-08ae5aecacef None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:49.001723 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff260b7-5c67-4854-9e94-08ae5aecacef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:49.003180 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff260b7-5c67-4854-9e94-08ae5aecacef None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:49.010013 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72299b-af69-4d31-ba9a-1e4ca21499ab None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:49.027044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 228/470] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:48 2022] POST /identity/v3/users => generated 248 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:49.028436 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba99065-e9cc-42ee-91ad-24fa3a7ef868 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:49.028728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba99065-e9cc-42ee-91ad-24fa3a7ef868 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:49.029010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba99065-e9cc-42ee-91ad-24fa3a7ef868 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:49.030521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 229/471] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:49.031266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba99065-e9cc-42ee-91ad-24fa3a7ef868 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:49.056714 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bdc5d-9788-43c3-861a-9b13b05a8274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:49.057397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bdc5d-9788-43c3-861a-9b13b05a8274 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:49.058305 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bdc5d-9788-43c3-861a-9b13b05a8274 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:49.058588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bdc5d-9788-43c3-861a-9b13b05a8274 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:49.059400 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bdc5d-9788-43c3-861a-9b13b05a8274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:49.059744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bdc5d-9788-43c3-861a-9b13b05a8274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:49.061176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bdc5d-9788-43c3-861a-9b13b05a8274 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:49.075690 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 243/472] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:49.077074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8a65e-b8c3-4745-a2ab-6ab20f8c005a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:49.077369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8a65e-b8c3-4745-a2ab-6ab20f8c005a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:49.077646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8a65e-b8c3-4745-a2ab-6ab20f8c005a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:49.103806 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1ab8a65e-b8c3-4745-a2ab-6ab20f8c005a None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:49.106407 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1ab8a65e-b8c3-4745-a2ab-6ab20f8c005a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:49.120746 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:49.189858 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba99065-e9cc-42ee-91ad-24fa3a7ef868 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:49.203721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 244/473] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:49.205021 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8a65e-b8c3-4745-a2ab-6ab20f8c005a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:49.238333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45efcd-0754-4b8e-80c9-d463ba6c2e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:49.238977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45efcd-0754-4b8e-80c9-d463ba6c2e08 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:49.239262 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45efcd-0754-4b8e-80c9-d463ba6c2e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:49.239539 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45efcd-0754-4b8e-80c9-d463ba6c2e08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:49.252642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45efcd-0754-4b8e-80c9-d463ba6c2e08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:49.254386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45efcd-0754-4b8e-80c9-d463ba6c2e08 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:49.270285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-0e45efcd-0754-4b8e-80c9-d463ba6c2e08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:49.270285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:49.270285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:49.270285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:49.270285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:49.270285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:49.270285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:49.270285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:49.270285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:49.270285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:49.270285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:49.270285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:49.270285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:49.270285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:49.270285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:49.270285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:49.271218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:49.272090 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:49.272090 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:49.272090 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:49.272090 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 230/474] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:49.279816 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45efcd-0754-4b8e-80c9-d463ba6c2e08 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:49.305317 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:49.323679 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8a65e-b8c3-4745-a2ab-6ab20f8c005a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:49.365376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a47722-b108-4249-b22d-1fd8feb9433e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:49.365576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a47722-b108-4249-b22d-1fd8feb9433e None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:49.365815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a47722-b108-4249-b22d-1fd8feb9433e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:49.366093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a47722-b108-4249-b22d-1fd8feb9433e None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:49.366742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a47722-b108-4249-b22d-1fd8feb9433e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:49.368953 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a47722-b108-4249-b22d-1fd8feb9433e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:49.397094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:49.406293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 245/475] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:49 2022] POST /identity/v3/users => generated 248 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:49.410762 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45efcd-0754-4b8e-80c9-d463ba6c2e08 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:49.457786 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28217c5-d982-4c0a-967c-dbd977f3d7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:49.458476 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28217c5-d982-4c0a-967c-dbd977f3d7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:49.458833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28217c5-d982-4c0a-967c-dbd977f3d7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:49.459176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28217c5-d982-4c0a-967c-dbd977f3d7e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:49.459949 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28217c5-d982-4c0a-967c-dbd977f3d7e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:49.460394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28217c5-d982-4c0a-967c-dbd977f3d7e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:49.461888 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28217c5-d982-4c0a-967c-dbd977f3d7e2 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:49.477143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 231/476] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:49.484608 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a47722-b108-4249-b22d-1fd8feb9433e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:49.509658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28217c5-d982-4c0a-967c-dbd977f3d7e2 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:49.627741 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:49.639363 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a47722-b108-4249-b22d-1fd8feb9433e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:49.654592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28217c5-d982-4c0a-967c-dbd977f3d7e2 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:49.681961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848329c1-8f39-4b28-ad38-cc9efc81cf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:49.682631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848329c1-8f39-4b28-ad38-cc9efc81cf3c None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:49.683455 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848329c1-8f39-4b28-ad38-cc9efc81cf3c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:49.683775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848329c1-8f39-4b28-ad38-cc9efc81cf3c None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:49.684429 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848329c1-8f39-4b28-ad38-cc9efc81cf3c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:49.686381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848329c1-8f39-4b28-ad38-cc9efc81cf3c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:49.696718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa630c-c7db-4e5b-8ab6-118e520d410c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:49.697712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa630c-c7db-4e5b-8ab6-118e520d410c None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:49.698822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa630c-c7db-4e5b-8ab6-118e520d410c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:49.699285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa630c-c7db-4e5b-8ab6-118e520d410c None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:49.700508 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa630c-c7db-4e5b-8ab6-118e520d410c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:49.702139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa630c-c7db-4e5b-8ab6-118e520d410c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:49.711499 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.sql.core [None req-8bfa630c-c7db-4e5b-8ab6-118e520d410c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 12:19:49.711499 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 12:19:49.711499 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [parameters: {'id': 'fbb063b56b1b4b96817da74059586970', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 12:19:49.711499 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81637) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 12:19:49.715657 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 246/477] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:49 2022] POST /identity/v3/users => generated 249 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:49.730044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-8bfa630c-c7db-4e5b-8ab6-118e520d410c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 12:19:49.730044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:49.730044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:19:49.730044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:19:49.730044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:19:49.730044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:19:49.730044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:19:49.730044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:19:49.730044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:19:49.730044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:19:49.730044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:19:49.730044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:19:49.730044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:19:49.730044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application result.read() Aug 24 12:19:49.730044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:19:49.731203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application yield resource Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise self.value Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application session.commit() Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 12:19:49.732386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self.session.flush() Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application compat.raise_( Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 12:19:49.733444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 12:19:49.734794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 12:19:49.734794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 12:19:49.734794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 12:19:49.734794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 12:19:49.734794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 12:19:49.734794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 12:19:49.734794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 12:19:49.734794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 12:19:49.734794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 12:19:49.734794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 12:19:49.734794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 12:19:49.734794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 12:19:49.734794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 12:19:49.734794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 12:19:49.734794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:19:49.734794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception Aug 24 12:19:49.736100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:19:49.736100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:19:49.736100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:19:49.736100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:19:49.736100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:19:49.736100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:19:49.736100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:19:49.736100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:19:49.736100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:19:49.736100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:19:49.736100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:19:49.736100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application result.read() Aug 24 12:19:49.736100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:19:49.736100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:19:49.736100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:19:49.736100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:19:49.737742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:19:49.737742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:19:49.737742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:19:49.737742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:19:49.737742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 12:19:49.737742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 12:19:49.737742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [parameters: {'id': 'fbb063b56b1b4b96817da74059586970', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 12:19:49.737742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 12:19:49.737742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:49.737742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:19:49.737742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:49.737742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:49.737742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:49.737742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:49.737742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:49.737742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:49.739706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:49.739706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:49.739706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:49.739706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:49.739706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:49.739706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:49.739706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 12:19:49.739706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 12:19:49.739706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:49.739706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:49.739706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 12:19:49.739706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 12:19:49.739706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 12:19:49.739706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 12:19:49.739706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 12:19:49.739706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:49.743091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 232/478] 10.222.0.58 () {60 vars in 1131 bytes} [Wed Aug 24 12:19:49 2022] POST /identity/v3/roles => generated 144 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:19:49.750187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848329c1-8f39-4b28-ad38-cc9efc81cf3c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:49.818329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:49.871186 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848329c1-8f39-4b28-ad38-cc9efc81cf3c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:49.909062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa948aaf-26e5-460d-a1a5-6973af36343e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:49.910022 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa948aaf-26e5-460d-a1a5-6973af36343e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:49.910297 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa948aaf-26e5-460d-a1a5-6973af36343e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:49.910569 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa948aaf-26e5-460d-a1a5-6973af36343e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:49.922386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa948aaf-26e5-460d-a1a5-6973af36343e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:49.927732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-aa948aaf-26e5-460d-a1a5-6973af36343e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:49.927732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:49.927732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:49.927732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:49.927732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:49.927732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:49.927732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:49.927732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:49.927732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:49.927732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:49.927732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:49.927732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:49.927732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:49.927732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:49.927732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:49.927732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:49.928613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:49.928613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:49.928613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:49.928613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:49.928613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:49.928613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:49.928613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:49.928613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:49.928613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:49.928613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:49.928613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:49.928613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:49.928613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:49.928613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:49.928613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:49.928613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:49.929928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:49.929928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:49.929928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:49.929928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:49.929928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:49.929928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:49.929928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:49.929928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:49.929928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:49.929928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:49.929928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 12:19:49.929928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:49.929928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 247/479] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:19:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:49.935948 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa630c-c7db-4e5b-8ab6-118e520d410c None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:50.042617 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa630c-c7db-4e5b-8ab6-118e520d410c None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:50.081591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53649e-0440-4d3d-8c4b-51803c6b163c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:50.082205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53649e-0440-4d3d-8c4b-51803c6b163c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:50.082483 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53649e-0440-4d3d-8c4b-51803c6b163c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:50.082751 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53649e-0440-4d3d-8c4b-51803c6b163c None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:50.083415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53649e-0440-4d3d-8c4b-51803c6b163c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:50.083743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53649e-0440-4d3d-8c4b-51803c6b163c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:50.085111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53649e-0440-4d3d-8c4b-51803c6b163c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:50.094079 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 233/480] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:50.998672 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014ee32-2ac4-4c36-b1b5-0efef440d29c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:50.999187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014ee32-2ac4-4c36-b1b5-0efef440d29c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:50.999187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014ee32-2ac4-4c36-b1b5-0efef440d29c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:51.000843 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 248/481] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:51.006207 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a525c15-ef4c-402f-91ab-f67ca397cf67 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:51.007080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a525c15-ef4c-402f-91ab-f67ca397cf67 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:51.007555 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a525c15-ef4c-402f-91ab-f67ca397cf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:51.021420 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:51.033559 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-1a525c15-ef4c-402f-91ab-f67ca397cf67 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:51.040281 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1a525c15-ef4c-402f-91ab-f67ca397cf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:51.127577 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 234/482] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:51.420018 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489a351-ee47-4c6d-b823-99b30d9a9080 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:51.420352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489a351-ee47-4c6d-b823-99b30d9a9080 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:51.420580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489a351-ee47-4c6d-b823-99b30d9a9080 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:51.422125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 249/483] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:51.427576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7777f-20b9-438f-afa4-110e0005e23e None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:51.427902 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7777f-20b9-438f-afa4-110e0005e23e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:51.428179 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7777f-20b9-438f-afa4-110e0005e23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:51.446891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-25f7777f-20b9-438f-afa4-110e0005e23e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:51.448157 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-25f7777f-20b9-438f-afa4-110e0005e23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:51.521709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 235/484] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:51.594116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489a351-ee47-4c6d-b823-99b30d9a9080 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:51.713413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489a351-ee47-4c6d-b823-99b30d9a9080 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:51.735141 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:51.759250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2284d-45e0-4707-ae2e-eb1c5272246d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:51.759962 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2284d-45e0-4707-ae2e-eb1c5272246d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:51.760243 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2284d-45e0-4707-ae2e-eb1c5272246d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:51.760515 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2284d-45e0-4707-ae2e-eb1c5272246d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:51.779507 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2284d-45e0-4707-ae2e-eb1c5272246d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:51.784427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-0bf2284d-45e0-4707-ae2e-eb1c5272246d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:19:51.784427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:51.784427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:51.784427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:51.784427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:51.784427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:51.784427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:51.784427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:51.784427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:51.784427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:51.784427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:51.784427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:51.784427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:51.784427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:51.784427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:51.784427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:51.785562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:51.785562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:51.785562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:51.785562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:51.785562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:51.785562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:51.785562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:51.785562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:51.785562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:51.785562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:51.785562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:51.785562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:51.785562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:51.785562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:51.785562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:51.785562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:51.786357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:51.786357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:51.786357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:51.786357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:51.786357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:51.786357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:51.786357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:51.786357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:51.786357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:51.786357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:51.786357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:19:51.786357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:51.786357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 250/485] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:19:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:51.793556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7777f-20b9-438f-afa4-110e0005e23e None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:51.868685 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:51.911189 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7777f-20b9-438f-afa4-110e0005e23e None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:51.947225 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f9ec44-95c2-40e5-97a0-d6c4d1227668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:51.947857 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9ec44-95c2-40e5-97a0-d6c4d1227668 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:51.948182 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9ec44-95c2-40e5-97a0-d6c4d1227668 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:51.948457 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f9ec44-95c2-40e5-97a0-d6c4d1227668 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:51.949125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9ec44-95c2-40e5-97a0-d6c4d1227668 None admin] RBAC: Adding query filter params (name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:51.949426 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9ec44-95c2-40e5-97a0-d6c4d1227668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:51.951056 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f9ec44-95c2-40e5-97a0-d6c4d1227668 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:51.959691 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 236/486] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:19:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:51.964916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2284d-45e0-4707-ae2e-eb1c5272246d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.022780 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53398f2-f333-42b1-9788-78abae3e9f80 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.023289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53398f2-f333-42b1-9788-78abae3e9f80 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.023363 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53398f2-f333-42b1-9788-78abae3e9f80 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.024968 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 237/487] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:52.029958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88472b07-3f5c-4f2c-bb66-3f0db53ffc71 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.030249 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88472b07-3f5c-4f2c-bb66-3f0db53ffc71 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.030526 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88472b07-3f5c-4f2c-bb66-3f0db53ffc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.049204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-88472b07-3f5c-4f2c-bb66-3f0db53ffc71 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:52.050418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-88472b07-3f5c-4f2c-bb66-3f0db53ffc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:52.113842 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2284d-45e0-4707-ae2e-eb1c5272246d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.141548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 238/488] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:52.142547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88472b07-3f5c-4f2c-bb66-3f0db53ffc71 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.157583 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4521f-e3cb-460f-b864-243b6c73a10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.158465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4521f-e3cb-460f-b864-243b6c73a10e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.158769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4521f-e3cb-460f-b864-243b6c73a10e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.159495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4521f-e3cb-460f-b864-243b6c73a10e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.175050 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4521f-e3cb-460f-b864-243b6c73a10e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:52.177400 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4521f-e3cb-460f-b864-243b6c73a10e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:52.196950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-fba4521f-e3cb-460f-b864-243b6c73a10e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:52.196950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:52.196950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:52.196950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:52.196950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:52.196950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:52.196950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:52.196950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:52.196950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:52.196950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:52.196950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:52.196950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:52.196950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:52.196950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:52.196950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:52.196950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:52.197915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:52.198797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:52.198797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:52.198797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:52.198797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 251/489] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:52.204843 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4521f-e3cb-460f-b864-243b6c73a10e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.223875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:52.265336 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:52.305530 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88472b07-3f5c-4f2c-bb66-3f0db53ffc71 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.370676 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4521f-e3cb-460f-b864-243b6c73a10e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.372818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a277b4-d4b5-480f-80c2-09cc46c26e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.373467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a277b4-d4b5-480f-80c2-09cc46c26e58 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.373746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a277b4-d4b5-480f-80c2-09cc46c26e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.374016 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a277b4-d4b5-480f-80c2-09cc46c26e58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.389470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a277b4-d4b5-480f-80c2-09cc46c26e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:52.400918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-23a277b4-d4b5-480f-80c2-09cc46c26e58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:19:52.400918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:52.400918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:52.400918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:52.400918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:52.400918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:52.400918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:52.400918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:52.400918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:52.400918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:52.400918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:52.400918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:52.400918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:52.400918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:52.400918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:52.400918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:52.402644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:52.402644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:52.402644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:52.402644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:52.402644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:52.402644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:52.402644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:52.402644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:52.402644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:52.402644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:52.402644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:52.402644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:52.402644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:52.402644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:52.402644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:52.402644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:52.403675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:52.403675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:52.403675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:52.403675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:52.403675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:52.403675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:52.403675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:52.403675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:52.403675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:52.403675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:52.403675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:19:52.403675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:52.403675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 239/490] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:52.407597 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51602b-7984-427a-9a24-595b8e322e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.408350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51602b-7984-427a-9a24-595b8e322e4a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.408733 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51602b-7984-427a-9a24-595b8e322e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.409142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51602b-7984-427a-9a24-595b8e322e4a None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.409886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51602b-7984-427a-9a24-595b8e322e4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:52.410291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51602b-7984-427a-9a24-595b8e322e4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:52.410891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a277b4-d4b5-480f-80c2-09cc46c26e58 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.411884 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51602b-7984-427a-9a24-595b8e322e4a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:52.425658 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 252/491] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:52.433306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51602b-7984-427a-9a24-595b8e322e4a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.516665 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a277b4-d4b5-480f-80c2-09cc46c26e58 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.534370 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:52.545471 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:52.554860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471fd71-b161-4570-b6f0-a5fe8f2e049e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.555472 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471fd71-b161-4570-b6f0-a5fe8f2e049e None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.555787 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471fd71-b161-4570-b6f0-a5fe8f2e049e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.556066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471fd71-b161-4570-b6f0-a5fe8f2e049e None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.556723 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471fd71-b161-4570-b6f0-a5fe8f2e049e None admin] RBAC: Adding query filter params (name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:52.557024 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471fd71-b161-4570-b6f0-a5fe8f2e049e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:52.559275 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471fd71-b161-4570-b6f0-a5fe8f2e049e None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:52.562762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51602b-7984-427a-9a24-595b8e322e4a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.572139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 240/492] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:19:52.572139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471fd71-b161-4570-b6f0-a5fe8f2e049e None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.600496 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be05ea7f-a346-4076-9a6e-c897f1331ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.601126 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05ea7f-a346-4076-9a6e-c897f1331ebb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.601408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05ea7f-a346-4076-9a6e-c897f1331ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.601682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be05ea7f-a346-4076-9a6e-c897f1331ebb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.609411 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05ea7f-a346-4076-9a6e-c897f1331ebb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:52.610855 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be05ea7f-a346-4076-9a6e-c897f1331ebb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:52.626256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-be05ea7f-a346-4076-9a6e-c897f1331ebb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:19:52.626256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:52.626256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:52.626256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:52.626256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:52.626256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:52.626256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:52.626256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:52.626256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:52.626256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:52.626256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:52.626256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:52.626256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:19:52.626256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:19:52.626256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:19:52.626256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:52.627215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:52.628144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:52.628144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:52.628144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:19:52.628144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:19:52.628144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:19:52.628144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:19:52.628144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:19:52.628144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:19:52.628144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:19:52.628144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:52.628144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 253/493] 10.222.0.58 () {56 vars in 1093 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:52.628721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451f307-3c15-4e9e-9035-e076a1a38569 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.629058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451f307-3c15-4e9e-9035-e076a1a38569 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.629231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451f307-3c15-4e9e-9035-e076a1a38569 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.634625 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 254/494] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:52.634625 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451f307-3c15-4e9e-9035-e076a1a38569 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.685504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471fd71-b161-4570-b6f0-a5fe8f2e049e None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.720861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48192842-fa27-427c-94ab-9d568b3aeff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.721567 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48192842-fa27-427c-94ab-9d568b3aeff2 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.721855 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48192842-fa27-427c-94ab-9d568b3aeff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.722130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48192842-fa27-427c-94ab-9d568b3aeff2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.732883 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48192842-fa27-427c-94ab-9d568b3aeff2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:52.738156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-48192842-fa27-427c-94ab-9d568b3aeff2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:19:52.738156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:52.738156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:52.738156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:52.738156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:52.738156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:52.738156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:52.738156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:52.738156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:52.738156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:52.738156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:52.738156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:52.738156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:52.738156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:52.738156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:52.738156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:52.739054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:52.739054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:52.739054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:52.739054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:52.739054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:52.739054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:52.739054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:52.739054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:52.739054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:52.739054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:52.739054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:52.739054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:52.739054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:52.739054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:52.739054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:52.739054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:52.740020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:52.740020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:52.740020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:52.740020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:52.740020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:52.740020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:52.740020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:52.740020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:52.740020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:52.740020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:52.740020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:19:52.740020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:52.740665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451f307-3c15-4e9e-9035-e076a1a38569 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.740744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 241/495] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:52.740744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48192842-fa27-427c-94ab-9d568b3aeff2 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.775695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a04b5c-6ccf-4656-aa64-fe10f0b8ab5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.776294 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a04b5c-6ccf-4656-aa64-fe10f0b8ab5c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.776565 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a04b5c-6ccf-4656-aa64-fe10f0b8ab5c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.776842 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a04b5c-6ccf-4656-aa64-fe10f0b8ab5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.778663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:52.787990 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a04b5c-6ccf-4656-aa64-fe10f0b8ab5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:52.789462 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a04b5c-6ccf-4656-aa64-fe10f0b8ab5c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:52.796664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:52.805235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-83a04b5c-6ccf-4656-aa64-fe10f0b8ab5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:52.805235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:52.805235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:52.805235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:52.805235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:52.805235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:52.805235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:52.805235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:52.805235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:52.805235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:52.805235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:52.805235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:52.805235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:52.805235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:52.805235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:52.805235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:52.806312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:52.807175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:52.807175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:52.807175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:52.807175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 255/496] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:52.808328 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b29d48-b3c0-445c-b348-40d865153210 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.808447 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b29d48-b3c0-445c-b348-40d865153210 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.808659 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b29d48-b3c0-445c-b348-40d865153210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.826121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-58b29d48-b3c0-445c-b348-40d865153210 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:52.827317 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-58b29d48-b3c0-445c-b348-40d865153210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:52.845775 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48192842-fa27-427c-94ab-9d568b3aeff2 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:52.886003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b79ed3-97b5-4f1c-ba40-48f31dc5841e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:52.886611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b79ed3-97b5-4f1c-ba40-48f31dc5841e None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.886891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b79ed3-97b5-4f1c-ba40-48f31dc5841e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.887169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b79ed3-97b5-4f1c-ba40-48f31dc5841e None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.887929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b79ed3-97b5-4f1c-ba40-48f31dc5841e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:52.888233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b79ed3-97b5-4f1c-ba40-48f31dc5841e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:52.889668 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b79ed3-97b5-4f1c-ba40-48f31dc5841e None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:52.896871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 256/497] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:52.898142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191186c-7cb9-4469-bb27-fff02f062601 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:52.898423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191186c-7cb9-4469-bb27-fff02f062601 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:52.898688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191186c-7cb9-4469-bb27-fff02f062601 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:52.900115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 257/498] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:52.904514 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191186c-7cb9-4469-bb27-fff02f062601 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:52.928433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 242/499] 10.222.0.58 () {56 vars in 1133 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:52.929267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b79ed3-97b5-4f1c-ba40-48f31dc5841e None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:53.020973 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191186c-7cb9-4469-bb27-fff02f062601 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:53.031346 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:53.038470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b79ed3-97b5-4f1c-ba40-48f31dc5841e None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:53.048691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:53.057037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7718e-0e63-492a-9e18-30bf995091a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:53.057668 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7718e-0e63-492a-9e18-30bf995091a5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:53.057952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7718e-0e63-492a-9e18-30bf995091a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:53.058267 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7718e-0e63-492a-9e18-30bf995091a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:53.059014 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7718e-0e63-492a-9e18-30bf995091a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:53.059248 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7718e-0e63-492a-9e18-30bf995091a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:53.060678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e7718e-0e63-492a-9e18-30bf995091a5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:53.069463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 258/500] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:53.070639 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ae459-7afe-4660-b6e9-6d047d69be94 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:53.070927 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ae459-7afe-4660-b6e9-6d047d69be94 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:53.071202 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ae459-7afe-4660-b6e9-6d047d69be94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:53.082569 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda0575-4b48-4dd6-b53a-7bad23c15c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:53.083472 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda0575-4b48-4dd6-b53a-7bad23c15c27 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:53.083994 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda0575-4b48-4dd6-b53a-7bad23c15c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:53.084331 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda0575-4b48-4dd6-b53a-7bad23c15c27 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:53.084984 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda0575-4b48-4dd6-b53a-7bad23c15c27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:53.085289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda0575-4b48-4dd6-b53a-7bad23c15c27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:53.086658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda0575-4b48-4dd6-b53a-7bad23c15c27 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:53.090285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-600ae459-7afe-4660-b6e9-6d047d69be94 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:53.091478 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-600ae459-7afe-4660-b6e9-6d047d69be94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:53.103774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 243/501] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:53.103774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda0575-4b48-4dd6-b53a-7bad23c15c27 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:53.166690 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 259/502] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:53.173548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ae459-7afe-4660-b6e9-6d047d69be94 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:53.226439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda0575-4b48-4dd6-b53a-7bad23c15c27 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:53.269425 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ee0b9-bb11-4aa1-885f-364082738612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:53.270304 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ee0b9-bb11-4aa1-885f-364082738612 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:53.270587 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ee0b9-bb11-4aa1-885f-364082738612 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:53.270861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ee0b9-bb11-4aa1-885f-364082738612 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:53.274586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ae459-7afe-4660-b6e9-6d047d69be94 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:53.297924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:53.299164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:53.313754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f41d7-f2cd-45f8-80b2-483a58ccb583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:53.314392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f41d7-f2cd-45f8-80b2-483a58ccb583 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:53.314671 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f41d7-f2cd-45f8-80b2-483a58ccb583 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:53.314943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f41d7-f2cd-45f8-80b2-483a58ccb583 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:53.326538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f41d7-f2cd-45f8-80b2-483a58ccb583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:53.328026 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f41d7-f2cd-45f8-80b2-483a58ccb583 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:53.343882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-952f41d7-f2cd-45f8-80b2-483a58ccb583 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:53.343882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:53.343882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:53.343882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:53.343882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:53.343882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:53.343882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:53.343882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:53.343882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:53.343882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:53.343882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:53.343882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:53.343882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:53.343882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:53.343882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:53.343882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:53.344822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:53.345681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:53.345681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:53.345681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:53.345681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 260/503] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:53.345930 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f41d7-f2cd-45f8-80b2-483a58ccb583 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:53.355725 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ee0b9-bb11-4aa1-885f-364082738612 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:53.357217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273ee0b9-bb11-4aa1-885f-364082738612 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:53.452488 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f41d7-f2cd-45f8-80b2-483a58ccb583 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:53.458588 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 244/504] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:53.460024 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ee0b9-bb11-4aa1-885f-364082738612 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:53.486399 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ca60d-0ce4-4808-a721-d7b93541e2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:53.487025 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ca60d-0ce4-4808-a721-d7b93541e2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:53.487304 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ca60d-0ce4-4808-a721-d7b93541e2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:53.487578 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ca60d-0ce4-4808-a721-d7b93541e2f5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:53.495098 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ca60d-0ce4-4808-a721-d7b93541e2f5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:53.496587 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ca60d-0ce4-4808-a721-d7b93541e2f5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:53.508007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-462ca60d-0ce4-4808-a721-d7b93541e2f5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:19:53.508007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:53.508007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:53.508007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:53.508007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:53.508007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:53.508007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:53.508007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:53.508007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:53.508007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:53.508007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:53.508007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:53.508007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:19:53.508007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:19:53.508007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:19:53.508007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:53.509153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:53.510311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:53.510311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:53.510311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:19:53.510311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:19:53.510311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:19:53.510311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:19:53.510311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:19:53.510311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:19:53.510311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:19:53.510311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:53.510311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 261/505] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:53.510311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ca60d-0ce4-4808-a721-d7b93541e2f5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:53.521586 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:53.542692 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:53.565920 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273ee0b9-bb11-4aa1-885f-364082738612 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:53.599588 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81444ce5-8873-4e2d-a041-b8da7d06f9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:53.600240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81444ce5-8873-4e2d-a041-b8da7d06f9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:53.600518 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81444ce5-8873-4e2d-a041-b8da7d06f9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:53.600787 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81444ce5-8873-4e2d-a041-b8da7d06f9e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:53.610958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81444ce5-8873-4e2d-a041-b8da7d06f9e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:53.611460 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ca60d-0ce4-4808-a721-d7b93541e2f5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:53.615658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-81444ce5-8873-4e2d-a041-b8da7d06f9e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:19:53.615658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:53.615658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:53.615658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:53.615658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:53.615658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:53.615658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:53.615658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:53.615658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:53.615658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:53.615658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:53.615658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:53.615658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:53.615658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:53.615658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:53.615658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:53.616595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:53.616595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:53.616595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:53.616595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:53.616595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:53.616595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:53.616595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:53.616595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:53.616595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:53.616595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:53.616595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:53.616595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:53.616595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:53.616595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:53.616595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:53.616595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:53.617415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:53.617415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:53.617415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:53.617415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:53.617415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:53.617415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:53.617415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:53.617415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:53.617415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:53.617415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:53.617415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:19:53.617415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:53.617415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 245/506] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:19:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:53.618087 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81444ce5-8873-4e2d-a041-b8da7d06f9e2 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:53.645092 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcc50d-f449-453d-8752-31d719cb9da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:53.645694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcc50d-f449-453d-8752-31d719cb9da1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:53.645974 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcc50d-f449-453d-8752-31d719cb9da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:53.646249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcc50d-f449-453d-8752-31d719cb9da1 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:53.646886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbcc50d-f449-453d-8752-31d719cb9da1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:53.647189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbcc50d-f449-453d-8752-31d719cb9da1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:53.648593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbcc50d-f449-453d-8752-31d719cb9da1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:53.660664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 262/507] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:53.661302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcc50d-f449-453d-8752-31d719cb9da1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:53.722958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81444ce5-8873-4e2d-a041-b8da7d06f9e2 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:53.739780 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:53.750791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:53.759271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190e511-3ab3-4068-a64c-12f819fd72d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:53.759892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcc50d-f449-453d-8752-31d719cb9da1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:53.760000 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e511-3ab3-4068-a64c-12f819fd72d8 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:53.760201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e511-3ab3-4068-a64c-12f819fd72d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:53.760495 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190e511-3ab3-4068-a64c-12f819fd72d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:53.770900 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190e511-3ab3-4068-a64c-12f819fd72d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:53.772404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190e511-3ab3-4068-a64c-12f819fd72d8 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:53.786658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-f190e511-3ab3-4068-a64c-12f819fd72d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:53.786658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:53.786658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:53.786658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:53.786658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:53.786658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:53.786658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:53.786658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:53.786658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:53.786658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:53.786658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:53.786658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:53.786658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:53.786658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:53.786658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:53.786658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:53.787864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:53.788925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:53.788925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:53.788925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:53.788925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 246/508] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:53.792478 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190e511-3ab3-4068-a64c-12f819fd72d8 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:53.795198 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bc9fc-8018-43d1-a8d7-9485b9f3bbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:53.796603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc9fc-8018-43d1-a8d7-9485b9f3bbd7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:53.796893 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc9fc-8018-43d1-a8d7-9485b9f3bbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:53.797168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc9fc-8018-43d1-a8d7-9485b9f3bbd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:53.808478 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bc9fc-8018-43d1-a8d7-9485b9f3bbd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:53.809911 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bc9fc-8018-43d1-a8d7-9485b9f3bbd7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:53.825063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-cc3bc9fc-8018-43d1-a8d7-9485b9f3bbd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:53.825063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:53.825063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:53.825063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:53.825063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:53.825063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:53.825063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:53.825063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:53.825063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:53.825063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:53.825063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:53.825063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:53.825063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:53.825063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:53.825063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:53.825063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:53.826172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:53.827103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:53.827103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:53.827103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:53.827103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 263/509] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:53.827103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bc9fc-8018-43d1-a8d7-9485b9f3bbd7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:53.898920 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190e511-3ab3-4068-a64c-12f819fd72d8 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:53.925699 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bc9fc-8018-43d1-a8d7-9485b9f3bbd7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:53.931646 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1ff4b-eb41-4179-9e07-4786cb00ed54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:53.932259 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1ff4b-eb41-4179-9e07-4786cb00ed54 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:53.933053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1ff4b-eb41-4179-9e07-4786cb00ed54 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:53.933383 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1ff4b-eb41-4179-9e07-4786cb00ed54 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:53.934137 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1ff4b-eb41-4179-9e07-4786cb00ed54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:53.934447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1ff4b-eb41-4179-9e07-4786cb00ed54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:53.935912 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1ff4b-eb41-4179-9e07-4786cb00ed54 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:53.953240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 247/510] 10.222.0.58 () {56 vars in 1137 bytes} [Wed Aug 24 12:19:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:53.954034 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1ff4b-eb41-4179-9e07-4786cb00ed54 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:53.961569 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564f36b-c583-4c79-b432-6e4760cdd1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:53.962174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564f36b-c583-4c79-b432-6e4760cdd1ff None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:53.962458 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564f36b-c583-4c79-b432-6e4760cdd1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:53.962728 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564f36b-c583-4c79-b432-6e4760cdd1ff None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:53.963385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564f36b-c583-4c79-b432-6e4760cdd1ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:53.963758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564f36b-c583-4c79-b432-6e4760cdd1ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:53.965504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564f36b-c583-4c79-b432-6e4760cdd1ff None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:53.973457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 264/511] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:19:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:53.974422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564f36b-c583-4c79-b432-6e4760cdd1ff None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:53.981534 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:53.986535 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:54.054595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1ff4b-eb41-4179-9e07-4786cb00ed54 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:54.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564f36b-c583-4c79-b432-6e4760cdd1ff None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:54.088933 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067bea2-68f7-47eb-8976-7ce0c39d58b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:54.089557 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067bea2-68f7-47eb-8976-7ce0c39d58b9 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:54.089835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067bea2-68f7-47eb-8976-7ce0c39d58b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:54.090102 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067bea2-68f7-47eb-8976-7ce0c39d58b9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:54.097086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067bea2-68f7-47eb-8976-7ce0c39d58b9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:54.098504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067bea2-68f7-47eb-8976-7ce0c39d58b9 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:54.111431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-3067bea2-68f7-47eb-8976-7ce0c39d58b9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:19:54.111431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:54.111431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:54.111431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:54.111431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:54.111431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:54.111431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:54.111431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:54.111431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:54.111431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:54.111431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:54.111431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:54.111431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:19:54.111431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:19:54.111431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:19:54.111431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:54.112389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:54.113284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:54.113284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:54.113284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:19:54.113284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:19:54.113284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:19:54.113284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:19:54.113284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:19:54.113284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:19:54.113284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:19:54.113284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:54.113284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 248/512] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:19:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:54.113284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067bea2-68f7-47eb-8976-7ce0c39d58b9 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:54.118548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d5f44-0a7c-4d06-9731-fb5222a971f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:54.119158 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d5f44-0a7c-4d06-9731-fb5222a971f5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:54.119445 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d5f44-0a7c-4d06-9731-fb5222a971f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:54.119753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d5f44-0a7c-4d06-9731-fb5222a971f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:54.121010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d5f44-0a7c-4d06-9731-fb5222a971f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:54.121316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d5f44-0a7c-4d06-9731-fb5222a971f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:54.122684 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5d5f44-0a7c-4d06-9731-fb5222a971f5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:54.134957 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 265/513] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:54.136110 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d5f44-0a7c-4d06-9731-fb5222a971f5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:54.202757 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:54.204172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:54.219486 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067bea2-68f7-47eb-8976-7ce0c39d58b9 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:54.240858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d5f44-0a7c-4d06-9731-fb5222a971f5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:54.253852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac821de-5dd2-44aa-b2e4-94f1e3768e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:54.254451 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac821de-5dd2-44aa-b2e4-94f1e3768e8d None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:54.254732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac821de-5dd2-44aa-b2e4-94f1e3768e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:54.255009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac821de-5dd2-44aa-b2e4-94f1e3768e8d None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:54.255662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac821de-5dd2-44aa-b2e4-94f1e3768e8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:54.255966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac821de-5dd2-44aa-b2e4-94f1e3768e8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:54.257349 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac821de-5dd2-44aa-b2e4-94f1e3768e8d None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:54.269036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 249/514] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:54.269838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac821de-5dd2-44aa-b2e4-94f1e3768e8d None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:54.279198 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428ac2f-eb88-4532-b16f-eb697a419708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:54.279840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428ac2f-eb88-4532-b16f-eb697a419708 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:54.280123 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428ac2f-eb88-4532-b16f-eb697a419708 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:54.280395 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428ac2f-eb88-4532-b16f-eb697a419708 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:54.349699 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428ac2f-eb88-4532-b16f-eb697a419708 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:54.351074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428ac2f-eb88-4532-b16f-eb697a419708 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:54.366390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac821de-5dd2-44aa-b2e4-94f1e3768e8d None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:54.401673 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca284c-d3ba-4fb1-b259-d8f7d3d0ed3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:54.402295 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca284c-d3ba-4fb1-b259-d8f7d3d0ed3f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:54.402577 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca284c-d3ba-4fb1-b259-d8f7d3d0ed3f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:54.402846 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca284c-d3ba-4fb1-b259-d8f7d3d0ed3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:54.412623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca284c-d3ba-4fb1-b259-d8f7d3d0ed3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:54.413756 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:54.414107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca284c-d3ba-4fb1-b259-d8f7d3d0ed3f None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:54.421671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:54.430610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-93ca284c-d3ba-4fb1-b259-d8f7d3d0ed3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:54.430610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:54.430610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:54.430610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:54.430610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:54.430610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:54.430610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:54.430610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:54.430610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:54.430610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:54.430610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:54.430610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:54.430610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:54.430610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:54.430610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:54.430610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:54.431547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:54.432516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:54.432516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:54.432516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:54.432516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 250/515] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:54.432966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca284c-d3ba-4fb1-b259-d8f7d3d0ed3f None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:54.451486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 266/516] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:54.452502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428ac2f-eb88-4532-b16f-eb697a419708 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:54.533598 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca284c-d3ba-4fb1-b259-d8f7d3d0ed3f None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:54.550314 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428ac2f-eb88-4532-b16f-eb697a419708 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:54.566815 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635464f3-803a-4be9-a056-abb7d82123e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:54.567417 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635464f3-803a-4be9-a056-abb7d82123e4 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:54.567794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635464f3-803a-4be9-a056-abb7d82123e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:54.568069 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635464f3-803a-4be9-a056-abb7d82123e4 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:54.568797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635464f3-803a-4be9-a056-abb7d82123e4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:54.569104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635464f3-803a-4be9-a056-abb7d82123e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:54.570536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635464f3-803a-4be9-a056-abb7d82123e4 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:54.583023 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88949d1-5f81-4015-9abb-335a15057ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:54.583635 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88949d1-5f81-4015-9abb-335a15057ef9 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:54.583918 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88949d1-5f81-4015-9abb-335a15057ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:54.584189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88949d1-5f81-4015-9abb-335a15057ef9 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:54.584817 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88949d1-5f81-4015-9abb-335a15057ef9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:54.586231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88949d1-5f81-4015-9abb-335a15057ef9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:54.587680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 251/517] 10.222.0.58 () {56 vars in 1137 bytes} [Wed Aug 24 12:19:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:54.588480 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635464f3-803a-4be9-a056-abb7d82123e4 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:54.608307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 267/518] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:19:54 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:19:54.609049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88949d1-5f81-4015-9abb-335a15057ef9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:54.655269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:54.656324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:54.707864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635464f3-803a-4be9-a056-abb7d82123e4 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:54.749204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb66bb7-6eb4-4c0e-a715-20c05a2d5a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:54.749550 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88949d1-5f81-4015-9abb-335a15057ef9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:54.749847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb66bb7-6eb4-4c0e-a715-20c05a2d5a9b None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:54.750128 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb66bb7-6eb4-4c0e-a715-20c05a2d5a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:54.750402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb66bb7-6eb4-4c0e-a715-20c05a2d5a9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:54.758501 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb66bb7-6eb4-4c0e-a715-20c05a2d5a9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:54.760045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb66bb7-6eb4-4c0e-a715-20c05a2d5a9b None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:54.773529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-efb66bb7-6eb4-4c0e-a715-20c05a2d5a9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:19:54.773529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:54.773529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:54.773529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:54.773529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:54.773529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:54.773529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:54.773529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:54.773529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:54.773529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:54.773529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:54.773529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:54.773529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:19:54.773529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:19:54.773529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:54.774487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:54.775342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:19:54.775342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:19:54.775342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:19:54.775342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:19:54.775342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:19:54.775342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:19:54.775342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:19:54.775342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:54.775342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 252/519] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:54.775956 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb66bb7-6eb4-4c0e-a715-20c05a2d5a9b None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:54.797519 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0061186-1561-48cf-9b54-874a59e7d001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:54.798377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0061186-1561-48cf-9b54-874a59e7d001 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:54.798503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0061186-1561-48cf-9b54-874a59e7d001 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:54.798711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0061186-1561-48cf-9b54-874a59e7d001 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:54.799360 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0061186-1561-48cf-9b54-874a59e7d001 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:54.799784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0061186-1561-48cf-9b54-874a59e7d001 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:54.801175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0061186-1561-48cf-9b54-874a59e7d001 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:54.813714 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 268/520] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:54.814703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0061186-1561-48cf-9b54-874a59e7d001 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:54.893236 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb66bb7-6eb4-4c0e-a715-20c05a2d5a9b None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:54.910929 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:54.912990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:54.931232 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60962e8-42ba-4638-97bc-de1000e47aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:54.931896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60962e8-42ba-4638-97bc-de1000e47aeb None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:54.932181 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60962e8-42ba-4638-97bc-de1000e47aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:54.933039 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60962e8-42ba-4638-97bc-de1000e47aeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:54.933510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0061186-1561-48cf-9b54-874a59e7d001 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:54.945617 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60962e8-42ba-4638-97bc-de1000e47aeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:54.947094 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60962e8-42ba-4638-97bc-de1000e47aeb None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:54.962869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-c60962e8-42ba-4638-97bc-de1000e47aeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:54.962869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:54.962869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:54.962869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:54.962869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:54.962869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:54.962869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:54.962869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:54.962869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:54.962869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:54.962869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:54.962869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:54.962869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:54.962869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:54.962869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:54.962869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:54.963860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:54.964721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:54.964721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:54.964721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:54.964721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 253/521] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:54.965390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60962e8-42ba-4638-97bc-de1000e47aeb None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:54.974045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42dc7b-1705-4126-ad8c-c121ab41eec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:54.974660 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42dc7b-1705-4126-ad8c-c121ab41eec0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:54.974943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42dc7b-1705-4126-ad8c-c121ab41eec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:54.975216 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42dc7b-1705-4126-ad8c-c121ab41eec0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:55.050519 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42dc7b-1705-4126-ad8c-c121ab41eec0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:55.052163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42dc7b-1705-4126-ad8c-c121ab41eec0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:55.075141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60962e8-42ba-4638-97bc-de1000e47aeb None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:55.110801 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1556d-2a88-49cf-9fe2-5f3a8f31f1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:55.111408 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1556d-2a88-49cf-9fe2-5f3a8f31f1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:55.111833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1556d-2a88-49cf-9fe2-5f3a8f31f1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:55.112110 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd1556d-2a88-49cf-9fe2-5f3a8f31f1b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:55.112969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1556d-2a88-49cf-9fe2-5f3a8f31f1b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:55.113291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1556d-2a88-49cf-9fe2-5f3a8f31f1b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:55.114869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd1556d-2a88-49cf-9fe2-5f3a8f31f1b7 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:55.139019 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 254/522] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:19:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:55.139019 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1556d-2a88-49cf-9fe2-5f3a8f31f1b7 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:55.144622 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:55.167605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 269/523] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:55.168474 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42dc7b-1705-4126-ad8c-c121ab41eec0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:55.169233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:55.252496 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd1556d-2a88-49cf-9fe2-5f3a8f31f1b7 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:55.276735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42dc7b-1705-4126-ad8c-c121ab41eec0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:55.290346 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551919cc-24f4-4cac-9897-39f8a1535cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:55.290982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551919cc-24f4-4cac-9897-39f8a1535cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:55.291267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551919cc-24f4-4cac-9897-39f8a1535cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:55.291541 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551919cc-24f4-4cac-9897-39f8a1535cf4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:55.299702 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551919cc-24f4-4cac-9897-39f8a1535cf4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:55.301320 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551919cc-24f4-4cac-9897-39f8a1535cf4 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:55.313123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-551919cc-24f4-4cac-9897-39f8a1535cf4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:19:55.313123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:55.313123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:55.313123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:55.313123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:55.313123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:55.313123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:55.313123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:55.313123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:55.313123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:55.313123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:55.313123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:55.313123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:19:55.313123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:19:55.313123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:19:55.313123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:55.314037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:55.315113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:55.315113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:55.315113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:19:55.315113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:19:55.315113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:19:55.315113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:19:55.315113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:19:55.315113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:19:55.315113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:19:55.315113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:55.315644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720740f-b72d-4273-99f5-539b902e4a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:55.315735 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 255/524] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:19:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:55.315820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720740f-b72d-4273-99f5-539b902e4a6f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:55.315820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720740f-b72d-4273-99f5-539b902e4a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:55.315820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720740f-b72d-4273-99f5-539b902e4a6f None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:55.316007 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551919cc-24f4-4cac-9897-39f8a1535cf4 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:55.316081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720740f-b72d-4273-99f5-539b902e4a6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:55.316081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720740f-b72d-4273-99f5-539b902e4a6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:55.317530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720740f-b72d-4273-99f5-539b902e4a6f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:55.334997 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 270/525] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:55.334997 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720740f-b72d-4273-99f5-539b902e4a6f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:55.379561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:55.414452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:55.427217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551919cc-24f4-4cac-9897-39f8a1535cf4 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:55.438942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720740f-b72d-4273-99f5-539b902e4a6f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:55.462414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0637b8-2016-4056-95c7-b3c21530f8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:55.463098 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0637b8-2016-4056-95c7-b3c21530f8ba None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:55.463450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0637b8-2016-4056-95c7-b3c21530f8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:55.463857 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0637b8-2016-4056-95c7-b3c21530f8ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:55.464767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0637b8-2016-4056-95c7-b3c21530f8ba None admin] RBAC: Adding query filter params (role.id=5a12926bd458420e81e72ecf825d67ae, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=311dbed3ab0b44a7830a93227ff9ad33) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:55.465080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0637b8-2016-4056-95c7-b3c21530f8ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:55.466520 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0637b8-2016-4056-95c7-b3c21530f8ba None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:55.472376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 256/526] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:55 2022] GET /identity/v3/role_assignments?user.id=311dbed3ab0b44a7830a93227ff9ad33&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=5a12926bd458420e81e72ecf825d67ae => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:55.473150 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0637b8-2016-4056-95c7-b3c21530f8ba None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:55.485616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1bd9b-6d00-4ea0-94cc-8a48c928de82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:55.487035 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1bd9b-6d00-4ea0-94cc-8a48c928de82 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:55.487793 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1bd9b-6d00-4ea0-94cc-8a48c928de82 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:55.488222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d1bd9b-6d00-4ea0-94cc-8a48c928de82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:55.503998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1bd9b-6d00-4ea0-94cc-8a48c928de82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:55.506353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d1bd9b-6d00-4ea0-94cc-8a48c928de82 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:55.527160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-91d1bd9b-6d00-4ea0-94cc-8a48c928de82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:55.527160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:55.527160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:55.527160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:55.527160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:55.527160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:55.527160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:55.527160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:55.527160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:55.527160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:55.527160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:55.527160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:55.527160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:55.527160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:55.527160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:55.527160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:55.528342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:55.529211 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:55.529211 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:55.529211 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:55.533218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 271/527] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:55.534224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1bd9b-6d00-4ea0-94cc-8a48c928de82 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:55.618283 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0637b8-2016-4056-95c7-b3c21530f8ba None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:55.649247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:55.656972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d1bd9b-6d00-4ea0-94cc-8a48c928de82 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:55.662110 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008900b3-be71-49da-baff-189e9eff7a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:55.663000 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008900b3-be71-49da-baff-189e9eff7a79 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:55.663479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008900b3-be71-49da-baff-189e9eff7a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:55.664047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008900b3-be71-49da-baff-189e9eff7a79 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:55.665087 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008900b3-be71-49da-baff-189e9eff7a79 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:55.665603 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008900b3-be71-49da-baff-189e9eff7a79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:55.667962 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008900b3-be71-49da-baff-189e9eff7a79 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:55.691349 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 257/528] 10.222.0.58 () {56 vars in 1139 bytes} [Wed Aug 24 12:19:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:55.692654 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008900b3-be71-49da-baff-189e9eff7a79 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:55.692814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c482ae-b5c9-48f5-ba0f-42e47b96d2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:55.693566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c482ae-b5c9-48f5-ba0f-42e47b96d2cd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:55.693925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c482ae-b5c9-48f5-ba0f-42e47b96d2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:55.694245 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c482ae-b5c9-48f5-ba0f-42e47b96d2cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:55.702069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c482ae-b5c9-48f5-ba0f-42e47b96d2cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:55.703652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c482ae-b5c9-48f5-ba0f-42e47b96d2cd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:55.709687 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:55.716784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-b3c482ae-b5c9-48f5-ba0f-42e47b96d2cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:19:55.716784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:55.716784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:55.716784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:55.716784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:55.716784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:55.716784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:55.716784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:55.716784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:55.716784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:55.716784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:55.716784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:55.716784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:19:55.716784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:19:55.716784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:55.717645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:55.718494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:19:55.718494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:19:55.718494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:19:55.718494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:19:55.718494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:19:55.718494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:19:55.718494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:19:55.718494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:55.722942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 272/529] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:55.724245 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c482ae-b5c9-48f5-ba0f-42e47b96d2cd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:55.826730 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008900b3-be71-49da-baff-189e9eff7a79 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:55.855205 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c482ae-b5c9-48f5-ba0f-42e47b96d2cd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:55.862455 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453ad40-00a8-42e0-8908-a4fd3531345a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:55.863069 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453ad40-00a8-42e0-8908-a4fd3531345a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:55.863353 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453ad40-00a8-42e0-8908-a4fd3531345a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:55.863652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453ad40-00a8-42e0-8908-a4fd3531345a None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:55.864314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453ad40-00a8-42e0-8908-a4fd3531345a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:55.864621 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453ad40-00a8-42e0-8908-a4fd3531345a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:55.866016 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453ad40-00a8-42e0-8908-a4fd3531345a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:55.888708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 258/530] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:55.889635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453ad40-00a8-42e0-8908-a4fd3531345a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:55.896435 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5cd1c-710b-4921-b947-3cc1e2020a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:55.897148 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5cd1c-710b-4921-b947-3cc1e2020a76 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:55.897524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5cd1c-710b-4921-b947-3cc1e2020a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:55.897890 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5cd1c-710b-4921-b947-3cc1e2020a76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:55.933665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:55.974603 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:56.002099 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5cd1c-710b-4921-b947-3cc1e2020a76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:56.004234 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f5cd1c-710b-4921-b947-3cc1e2020a76 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:56.010589 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453ad40-00a8-42e0-8908-a4fd3531345a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:56.048608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04381793-9f0d-460f-8520-0d8f900ff4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:56.049375 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04381793-9f0d-460f-8520-0d8f900ff4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.049748 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04381793-9f0d-460f-8520-0d8f900ff4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.050114 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04381793-9f0d-460f-8520-0d8f900ff4a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.050914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04381793-9f0d-460f-8520-0d8f900ff4a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:56.051313 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04381793-9f0d-460f-8520-0d8f900ff4a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:56.052935 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04381793-9f0d-460f-8520-0d8f900ff4a5 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:56.066512 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 259/531] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:19:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:56.068548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04381793-9f0d-460f-8520-0d8f900ff4a5 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:56.129288 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 273/532] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:56.130561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5cd1c-710b-4921-b947-3cc1e2020a76 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:56.176787 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04381793-9f0d-460f-8520-0d8f900ff4a5 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:56.195108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:56.208204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:56.214597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5da85-3957-4797-9cd7-afbecd850cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:56.216020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5da85-3957-4797-9cd7-afbecd850cbe None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.216020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5da85-3957-4797-9cd7-afbecd850cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.216020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb5da85-3957-4797-9cd7-afbecd850cbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.223980 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5da85-3957-4797-9cd7-afbecd850cbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:56.225440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb5da85-3957-4797-9cd7-afbecd850cbe None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:56.237870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-5fb5da85-3957-4797-9cd7-afbecd850cbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:19:56.237870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:56.237870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:56.237870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:56.237870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:56.237870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:56.237870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:56.237870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:56.237870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:56.237870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:56.237870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:56.237870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:56.237870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:19:56.237870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:19:56.237870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:56.238939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:56.239944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:19:56.239944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:19:56.239944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:19:56.239944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:19:56.239944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:19:56.239944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:19:56.239944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:19:56.239944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:56.239944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 260/533] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:56.240416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5da85-3957-4797-9cd7-afbecd850cbe None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:56.253386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f5cd1c-710b-4921-b947-3cc1e2020a76 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:56.292925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba523a7a-f1c9-4fd7-a484-358a0a9d2369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:56.294330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba523a7a-f1c9-4fd7-a484-358a0a9d2369 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.294757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba523a7a-f1c9-4fd7-a484-358a0a9d2369 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.295189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba523a7a-f1c9-4fd7-a484-358a0a9d2369 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.296188 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba523a7a-f1c9-4fd7-a484-358a0a9d2369 None admin] RBAC: Adding query filter params (role.id=5a12926bd458420e81e72ecf825d67ae, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=dd8df2f3357d4660bc22366f9746f89c) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:56.296662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba523a7a-f1c9-4fd7-a484-358a0a9d2369 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:56.298356 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba523a7a-f1c9-4fd7-a484-358a0a9d2369 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:56.311991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 274/534] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:56 2022] GET /identity/v3/role_assignments?user.id=dd8df2f3357d4660bc22366f9746f89c&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=5a12926bd458420e81e72ecf825d67ae => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:56.313367 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba523a7a-f1c9-4fd7-a484-358a0a9d2369 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:56.347552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb5da85-3957-4797-9cd7-afbecd850cbe None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:56.383915 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a2817-f3bd-4513-b390-2a85658e4b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:56.384860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a2817-f3bd-4513-b390-2a85658e4b39 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.384860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a2817-f3bd-4513-b390-2a85658e4b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.385159 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a2817-f3bd-4513-b390-2a85658e4b39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.397646 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a2817-f3bd-4513-b390-2a85658e4b39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:56.399285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a2817-f3bd-4513-b390-2a85658e4b39 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:56.414970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-0c4a2817-f3bd-4513-b390-2a85658e4b39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:56.414970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:56.414970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:56.414970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:56.414970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:56.414970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:56.414970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:56.414970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:56.414970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:56.414970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:56.414970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:56.414970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:56.414970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:56.414970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:56.414970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:56.414970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:56.416164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:56.417143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:56.417143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:56.417143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:56.417143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 261/535] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:56.422403 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a2817-f3bd-4513-b390-2a85658e4b39 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:56.478689 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:56.520592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba523a7a-f1c9-4fd7-a484-358a0a9d2369 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:56.551223 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a2817-f3bd-4513-b390-2a85658e4b39 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:56.559825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:56.596864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b161e-c454-4e6e-8b80-35f2db4b9757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:56.598272 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b161e-c454-4e6e-8b80-35f2db4b9757 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.599148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b161e-c454-4e6e-8b80-35f2db4b9757 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.599926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b161e-c454-4e6e-8b80-35f2db4b9757 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.601300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b161e-c454-4e6e-8b80-35f2db4b9757 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:56.602109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c49e5-6fb8-4f81-8565-4caabca7b751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:56.602594 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b161e-c454-4e6e-8b80-35f2db4b9757 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:56.603353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c49e5-6fb8-4f81-8565-4caabca7b751 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.605045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b161e-c454-4e6e-8b80-35f2db4b9757 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:56.605508 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c49e5-6fb8-4f81-8565-4caabca7b751 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.605950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c49e5-6fb8-4f81-8565-4caabca7b751 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.606820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c49e5-6fb8-4f81-8565-4caabca7b751 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:56.609221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c49e5-6fb8-4f81-8565-4caabca7b751 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:56.610880 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c49e5-6fb8-4f81-8565-4caabca7b751 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:56.632728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 262/536] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:56.639776 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b161e-c454-4e6e-8b80-35f2db4b9757 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:56.645367 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 275/537] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:19:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:56.650996 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c49e5-6fb8-4f81-8565-4caabca7b751 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:56.753718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b161e-c454-4e6e-8b80-35f2db4b9757 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:56.755450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:56.792695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6a823-c612-4802-8846-d24447b6daf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:56.793337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6a823-c612-4802-8846-d24447b6daf3 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.793625 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6a823-c612-4802-8846-d24447b6daf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.793900 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6a823-c612-4802-8846-d24447b6daf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.794792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c49e5-6fb8-4f81-8565-4caabca7b751 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:56.801203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6a823-c612-4802-8846-d24447b6daf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:56.802882 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6a823-c612-4802-8846-d24447b6daf3 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:56.816245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-e4c6a823-c612-4802-8846-d24447b6daf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:19:56.816245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:56.816245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:56.816245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:56.816245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:56.816245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:56.816245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:56.816245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:56.816245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:56.816245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:56.816245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:56.816245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:56.816245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:19:56.816245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:19:56.816245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:56.817347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:56.818328 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:19:56.818328 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:19:56.818328 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:19:56.818328 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:19:56.818328 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:19:56.818328 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:19:56.818328 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:19:56.818328 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:56.818328 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 263/538] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:19:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:56.819014 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7dc47-450d-4a56-a709-ea0205dd4e06 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.819151 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7dc47-450d-4a56-a709-ea0205dd4e06 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.820405 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7dc47-450d-4a56-a709-ea0205dd4e06 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.825498 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 264/539] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:56.826845 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7dc47-450d-4a56-a709-ea0205dd4e06 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:56.838016 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a291fa-5c47-4393-960c-a40da36cff7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:56.839198 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a291fa-5c47-4393-960c-a40da36cff7e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.839747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a291fa-5c47-4393-960c-a40da36cff7e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.840242 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a291fa-5c47-4393-960c-a40da36cff7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.841194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a291fa-5c47-4393-960c-a40da36cff7e None admin] RBAC: Adding query filter params (role.id=5a12926bd458420e81e72ecf825d67ae, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=a42d8e462ba542a5bf58ba052dceeb8f) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:56.841702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a291fa-5c47-4393-960c-a40da36cff7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:56.843354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a291fa-5c47-4393-960c-a40da36cff7e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:56.849425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 276/540] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:56 2022] GET /identity/v3/role_assignments?user.id=a42d8e462ba542a5bf58ba052dceeb8f&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=5a12926bd458420e81e72ecf825d67ae => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:56.850956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c78671-7a2e-4a59-b896-4d11c7bae905 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.851475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c78671-7a2e-4a59-b896-4d11c7bae905 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.851975 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c78671-7a2e-4a59-b896-4d11c7bae905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.871502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a2c78671-7a2e-4a59-b896-4d11c7bae905 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:56.873136 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a2c78671-7a2e-4a59-b896-4d11c7bae905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:56.948565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7dc47-450d-4a56-a709-ea0205dd4e06 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:56.973347 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:56.989791 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490916a-84af-42b7-8152-efd75a185a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:56.990420 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490916a-84af-42b7-8152-efd75a185a85 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:56.990702 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490916a-84af-42b7-8152-efd75a185a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:56.990975 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490916a-84af-42b7-8152-efd75a185a85 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:56.991721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490916a-84af-42b7-8152-efd75a185a85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:56.992016 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490916a-84af-42b7-8152-efd75a185a85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:56.993494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0490916a-84af-42b7-8152-efd75a185a85 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:57.007212 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 265/541] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:19:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:57.017692 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490916a-84af-42b7-8152-efd75a185a85 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:57.048958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 277/542] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:57.065354 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:57.137871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490916a-84af-42b7-8152-efd75a185a85 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:57.177053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef7ddb2-f6bc-4248-b250-ac989985cf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:57.177707 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7ddb2-f6bc-4248-b250-ac989985cf46 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:57.177983 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7ddb2-f6bc-4248-b250-ac989985cf46 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:57.178256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7ddb2-f6bc-4248-b250-ac989985cf46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:57.178996 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef7ddb2-f6bc-4248-b250-ac989985cf46 None admin] RBAC: Adding query filter params (role.id=5a12926bd458420e81e72ecf825d67ae, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=34fcb5885325472bbe338cf05d3788f4) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:57.179305 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef7ddb2-f6bc-4248-b250-ac989985cf46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:57.180822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef7ddb2-f6bc-4248-b250-ac989985cf46 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:57.190630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 266/543] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:19:57 2022] GET /identity/v3/role_assignments?user.id=34fcb5885325472bbe338cf05d3788f4&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=5a12926bd458420e81e72ecf825d67ae => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:57.537510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c78671-7a2e-4a59-b896-4d11c7bae905 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:57.650630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bae8f-e611-4a0d-8a69-ff18e063ac78 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:57.650987 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bae8f-e611-4a0d-8a69-ff18e063ac78 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:57.651756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511bae8f-e611-4a0d-8a69-ff18e063ac78 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:57.653693 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 267/544] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:19:57.665058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c78671-7a2e-4a59-b896-4d11c7bae905 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:57.665228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c16912-126a-4aba-bb19-86029529db85 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:57.665446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c16912-126a-4aba-bb19-86029529db85 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:57.665665 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c16912-126a-4aba-bb19-86029529db85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:57.707394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-a4c16912-126a-4aba-bb19-86029529db85 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:57.708644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a4c16912-126a-4aba-bb19-86029529db85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:57.738069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4da573-4263-41e1-93bb-69b0ab4b3d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:57.738706 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4da573-4263-41e1-93bb-69b0ab4b3d78 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:57.738986 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4da573-4263-41e1-93bb-69b0ab4b3d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:57.739260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4da573-4263-41e1-93bb-69b0ab4b3d78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:57.763756 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4da573-4263-41e1-93bb-69b0ab4b3d78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:57.768517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6e4da573-4263-41e1-93bb-69b0ab4b3d78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:19:57.768517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:57.768517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:57.768517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:57.768517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:57.768517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:57.768517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:57.768517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:57.768517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:57.768517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:57.768517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:57.768517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:57.768517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:57.768517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:57.768517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:57.768517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:57.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:57.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:57.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:57.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:57.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:57.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:57.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:57.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:57.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:57.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:57.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:57.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:57.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:57.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:57.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:57.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:57.770227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:57.770227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:57.770227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:57.770227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:57.770227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:57.770227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:57.770227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:57.770227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:57.770227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:57.770227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:57.770227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:19:57.770227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:57.770227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 278/545] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:19:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:57.778108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4da573-4263-41e1-93bb-69b0ab4b3d78 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:57.837624 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 268/546] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:19:57.860949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:57.979920 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4da573-4263-41e1-93bb-69b0ab4b3d78 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:58.028346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22b84d-ac4e-472b-81ac-397a624f4a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:58.028967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22b84d-ac4e-472b-81ac-397a624f4a1f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:58.029252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22b84d-ac4e-472b-81ac-397a624f4a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:58.029526 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c22b84d-ac4e-472b-81ac-397a624f4a1f None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:58.030199 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22b84d-ac4e-472b-81ac-397a624f4a1f None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:58.030505 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22b84d-ac4e-472b-81ac-397a624f4a1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:58.031938 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c22b84d-ac4e-472b-81ac-397a624f4a1f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:58.040909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 279/547] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:19:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:58.047010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c16912-126a-4aba-bb19-86029529db85 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:58.075948 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:58.191215 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c16912-126a-4aba-bb19-86029529db85 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:58.210910 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22b84d-ac4e-472b-81ac-397a624f4a1f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:58.234298 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf358448-8951-445c-8ade-eae7ec0aa26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:58.235287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf358448-8951-445c-8ade-eae7ec0aa26c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:58.235899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf358448-8951-445c-8ade-eae7ec0aa26c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:58.240261 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf358448-8951-445c-8ade-eae7ec0aa26c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:58.261901 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf358448-8951-445c-8ade-eae7ec0aa26c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:58.264972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf358448-8951-445c-8ade-eae7ec0aa26c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:58.297282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-cf358448-8951-445c-8ade-eae7ec0aa26c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:58.297282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:58.297282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:58.297282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:58.297282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:58.297282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:58.297282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:58.297282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:58.297282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:58.297282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:58.297282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:58.297282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:58.297282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:58.297282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:58.297282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:58.297282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:58.298265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:58.299119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:58.299119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:58.299119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:58.308759 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 269/548] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:58.312505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf358448-8951-445c-8ade-eae7ec0aa26c None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:58.397390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:58.412516 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c22b84d-ac4e-472b-81ac-397a624f4a1f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:58.439278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:58.472369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27630d-1210-4b5b-adf4-574ef6463136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:58.473231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27630d-1210-4b5b-adf4-574ef6463136 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:58.473513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27630d-1210-4b5b-adf4-574ef6463136 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:58.473790 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27630d-1210-4b5b-adf4-574ef6463136 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:58.490384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e27630d-1210-4b5b-adf4-574ef6463136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:58.496681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-4e27630d-1210-4b5b-adf4-574ef6463136 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:19:58.496681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:58.496681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:58.496681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:58.496681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:58.496681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:58.496681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:58.496681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:58.496681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:58.496681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:58.496681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:58.496681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:58.496681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:58.496681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:58.496681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:58.496681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:58.497564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:58.497564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:58.497564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:58.497564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:58.497564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:58.497564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:58.497564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:58.497564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:58.497564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:58.497564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:58.497564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:58.497564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:58.497564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:58.497564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:58.497564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:58.497564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:58.498922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:58.498922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:58.498922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:58.498922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:58.498922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:58.498922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:58.498922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:58.498922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:58.498922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:58.498922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:58.498922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:19:58.498922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:58.498922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 280/549] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:19:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:58.503596 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf358448-8951-445c-8ade-eae7ec0aa26c None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:58.508992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27630d-1210-4b5b-adf4-574ef6463136 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:58.548835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f65af-933d-409e-b38c-cf01eb1b2588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:58.551339 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f65af-933d-409e-b38c-cf01eb1b2588 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:58.551645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f65af-933d-409e-b38c-cf01eb1b2588 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:58.552001 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f65af-933d-409e-b38c-cf01eb1b2588 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:58.552663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f65af-933d-409e-b38c-cf01eb1b2588 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:58.552971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f65af-933d-409e-b38c-cf01eb1b2588 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:58.557695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f65af-933d-409e-b38c-cf01eb1b2588 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:58.583681 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 270/550] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:58.590115 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f65af-933d-409e-b38c-cf01eb1b2588 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:58.667526 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27630d-1210-4b5b-adf4-574ef6463136 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:58.704155 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc39856-1d63-44a4-afbf-9ac11a1d8a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:58.704815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc39856-1d63-44a4-afbf-9ac11a1d8a84 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:58.705165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc39856-1d63-44a4-afbf-9ac11a1d8a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:58.705444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc39856-1d63-44a4-afbf-9ac11a1d8a84 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:58.706152 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc39856-1d63-44a4-afbf-9ac11a1d8a84 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:58.706464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc39856-1d63-44a4-afbf-9ac11a1d8a84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:58.707878 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc39856-1d63-44a4-afbf-9ac11a1d8a84 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:58.717638 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 281/551] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:19:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:58.718859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af2313-94a4-406f-acfa-b33cc9ffd94d None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:58.719210 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af2313-94a4-406f-acfa-b33cc9ffd94d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:58.719497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af2313-94a4-406f-acfa-b33cc9ffd94d None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:58.720727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 282/552] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:58.725537 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af2313-94a4-406f-acfa-b33cc9ffd94d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:58.740875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f65af-933d-409e-b38c-cf01eb1b2588 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:58.756255 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:58.779957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31468f4e-1bc5-4968-a737-f7cbc557cf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:58.780826 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31468f4e-1bc5-4968-a737-f7cbc557cf9d None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:58.781256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31468f4e-1bc5-4968-a737-f7cbc557cf9d None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:58.781772 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31468f4e-1bc5-4968-a737-f7cbc557cf9d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:58.791874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:58.792827 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31468f4e-1bc5-4968-a737-f7cbc557cf9d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:58.795284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31468f4e-1bc5-4968-a737-f7cbc557cf9d None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:58.811817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-31468f4e-1bc5-4968-a737-f7cbc557cf9d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:19:58.811817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:58.811817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:58.811817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:58.811817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:58.811817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:58.811817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:58.811817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:58.811817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:58.811817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:58.811817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:58.811817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:58.811817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:19:58.811817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:19:58.811817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:19:58.811817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:58.813053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:58.814273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:58.814273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:58.814273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:19:58.814273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:19:58.814273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:19:58.814273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:19:58.814273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:19:58.814273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:19:58.814273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:19:58.814273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:58.817450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 271/553] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:19:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:58.819083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146611fa-4193-4cfa-9d72-b30ff8848d38 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:58.819803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146611fa-4193-4cfa-9d72-b30ff8848d38 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:58.820222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146611fa-4193-4cfa-9d72-b30ff8848d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:58.844843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-146611fa-4193-4cfa-9d72-b30ff8848d38 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:58.845071 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af2313-94a4-406f-acfa-b33cc9ffd94d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:58.846729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-146611fa-4193-4cfa-9d72-b30ff8848d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:58.888478 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921f1d0-800b-4d9d-a28d-90ca51be68b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:58.890047 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921f1d0-800b-4d9d-a28d-90ca51be68b8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:58.890555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921f1d0-800b-4d9d-a28d-90ca51be68b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:58.890979 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921f1d0-800b-4d9d-a28d-90ca51be68b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:58.904688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921f1d0-800b-4d9d-a28d-90ca51be68b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:58.907041 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921f1d0-800b-4d9d-a28d-90ca51be68b8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:58.925991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-b921f1d0-800b-4d9d-a28d-90ca51be68b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:58.925991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:58.925991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:58.925991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:58.925991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:58.925991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:58.925991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:58.925991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:58.925991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:58.925991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:58.925991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:58.925991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:58.925991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:58.925991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:58.925991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:58.925991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:58.926978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:58.927897 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:58.927897 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:58.927897 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:19:58.929070 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 283/554] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:58.929721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921f1d0-800b-4d9d-a28d-90ca51be68b8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:58.937061 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 272/555] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:58.937928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146611fa-4193-4cfa-9d72-b30ff8848d38 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:59.016878 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:59.053394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921f1d0-800b-4d9d-a28d-90ca51be68b8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:59.073736 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146611fa-4193-4cfa-9d72-b30ff8848d38 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:59.089647 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1befa-e171-490e-ab8d-df4c3a9d3722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:59.090260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1befa-e171-490e-ab8d-df4c3a9d3722 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.090543 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1befa-e171-490e-ab8d-df4c3a9d3722 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.090964 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a1befa-e171-490e-ab8d-df4c3a9d3722 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.091740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1befa-e171-490e-ab8d-df4c3a9d3722 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:59.092052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1befa-e171-490e-ab8d-df4c3a9d3722 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:59.093506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a1befa-e171-490e-ab8d-df4c3a9d3722 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:59.097539 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:59.117534 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5626b8f1-8d62-4991-bb3a-1292636ff1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:59.118492 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626b8f1-8d62-4991-bb3a-1292636ff1af None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.118968 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626b8f1-8d62-4991-bb3a-1292636ff1af None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.119532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5626b8f1-8d62-4991-bb3a-1292636ff1af None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.121097 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626b8f1-8d62-4991-bb3a-1292636ff1af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:59.121681 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626b8f1-8d62-4991-bb3a-1292636ff1af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:59.124227 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5626b8f1-8d62-4991-bb3a-1292636ff1af None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:59.134868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 284/556] 10.222.0.58 () {56 vars in 1135 bytes} [Wed Aug 24 12:19:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:59.138643 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 273/557] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:59.143921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1befa-e171-490e-ab8d-df4c3a9d3722 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:59.145916 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5626b8f1-8d62-4991-bb3a-1292636ff1af None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:59.316856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a1befa-e171-490e-ab8d-df4c3a9d3722 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:59.320569 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5626b8f1-8d62-4991-bb3a-1292636ff1af None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:59.351582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:59.356509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f19204-1101-4aa4-a0e0-5cda0bceb886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:59.357334 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f19204-1101-4aa4-a0e0-5cda0bceb886 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.357609 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f19204-1101-4aa4-a0e0-5cda0bceb886 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.357892 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f19204-1101-4aa4-a0e0-5cda0bceb886 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.362923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fdeba-422a-48eb-9d16-807319840b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:59.363542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fdeba-422a-48eb-9d16-807319840b22 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.363854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fdeba-422a-48eb-9d16-807319840b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.364131 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930fdeba-422a-48eb-9d16-807319840b22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.366250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f19204-1101-4aa4-a0e0-5cda0bceb886 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:59.367729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f19204-1101-4aa4-a0e0-5cda0bceb886 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:59.379724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-c2f19204-1101-4aa4-a0e0-5cda0bceb886 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:19:59.379724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:59.379724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:59.379724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:59.379724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:59.379724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:59.379724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:59.379724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:59.379724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:59.379724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:59.379724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:59.379724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:59.379724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:19:59.379724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:19:59.379724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:19:59.379724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:59.381158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:59.382906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:59.382906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:59.382906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:19:59.382906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:19:59.382906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:19:59.382906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:19:59.382906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:19:59.382906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:19:59.382906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:19:59.382906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:59.382906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 274/558] 10.222.0.58 () {56 vars in 1093 bytes} [Wed Aug 24 12:19:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:59.382906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba5c4f-e192-4232-b8c9-a674764a354a None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.382906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba5c4f-e192-4232-b8c9-a674764a354a None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.383661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba5c4f-e192-4232-b8c9-a674764a354a None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.384233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 275/559] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:59.388497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba5c4f-e192-4232-b8c9-a674764a354a None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:59.444568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:59.451840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fdeba-422a-48eb-9d16-807319840b22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:59.453321 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930fdeba-422a-48eb-9d16-807319840b22 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:59.510719 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba5c4f-e192-4232-b8c9-a674764a354a None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:59.546821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40263e-087f-44dc-9a99-5541a2d1313b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:59.547456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40263e-087f-44dc-9a99-5541a2d1313b None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.547769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40263e-087f-44dc-9a99-5541a2d1313b None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.548091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40263e-087f-44dc-9a99-5541a2d1313b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.559673 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 285/560] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:19:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:19:59.559673 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fdeba-422a-48eb-9d16-807319840b22 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:59.561044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a40263e-087f-44dc-9a99-5541a2d1313b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:19:59.565788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-6a40263e-087f-44dc-9a99-5541a2d1313b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:19:59.565788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:59.565788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:59.565788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:59.565788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:59.565788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:59.565788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:59.565788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:59.565788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:59.565788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:59.565788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:59.565788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:59.565788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:19:59.565788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:19:59.565788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:19:59.565788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:19:59.566803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:19:59.566803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:19:59.566803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:19:59.566803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:19:59.566803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:59.566803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:59.566803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:59.566803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:59.566803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:59.566803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:59.566803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:59.566803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:59.566803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:59.566803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:59.566803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:59.566803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:59.567729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:59.567729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:59.567729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:59.567729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:59.567729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:19:59.567729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:19:59.567729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:19:59.567729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:19:59.567729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:19:59.567729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:19:59.567729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:19:59.567729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:59.567729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 276/561] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:19:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:59.568433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a021e55-2a8a-416b-8527-03d20da83e85 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.568433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a021e55-2a8a-416b-8527-03d20da83e85 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.568566 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a021e55-2a8a-416b-8527-03d20da83e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.585961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-2a021e55-2a8a-416b-8527-03d20da83e85 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:59.587161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2a021e55-2a8a-416b-8527-03d20da83e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:59.588469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:59.653684 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 277/562] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:59.663861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73ac8e-069e-4d30-80a8-3af4c6098199 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.663861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73ac8e-069e-4d30-80a8-3af4c6098199 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.663861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73ac8e-069e-4d30-80a8-3af4c6098199 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.663861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 278/563] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:19:59.663861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73ac8e-069e-4d30-80a8-3af4c6098199 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:59.664337 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930fdeba-422a-48eb-9d16-807319840b22 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:59.695983 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:59.702325 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801e1f1-062c-4769-8d7e-ae678c67dfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:59.703452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801e1f1-062c-4769-8d7e-ae678c67dfe5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.703768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801e1f1-062c-4769-8d7e-ae678c67dfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.704048 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2801e1f1-062c-4769-8d7e-ae678c67dfe5 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.704769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801e1f1-062c-4769-8d7e-ae678c67dfe5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:59.705078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801e1f1-062c-4769-8d7e-ae678c67dfe5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:59.706524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2801e1f1-062c-4769-8d7e-ae678c67dfe5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:59.724625 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 286/564] 10.222.0.58 () {56 vars in 1133 bytes} [Wed Aug 24 12:19:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:59.728715 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801e1f1-062c-4769-8d7e-ae678c67dfe5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:59.781919 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73ac8e-069e-4d30-80a8-3af4c6098199 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:59.817165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:19:59.818345 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f8878-d767-452f-95b9-45df0c5e5d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:59.819038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f8878-d767-452f-95b9-45df0c5e5d9f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.819217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f8878-d767-452f-95b9-45df0c5e5d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.819562 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f8878-d767-452f-95b9-45df0c5e5d9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.830579 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f8878-d767-452f-95b9-45df0c5e5d9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:59.832064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f8878-d767-452f-95b9-45df0c5e5d9f None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:59.835254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2801e1f1-062c-4769-8d7e-ae678c67dfe5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:19:59.847319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-a81f8878-d767-452f-95b9-45df0c5e5d9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:59.847319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:19:59.847319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:19:59.847319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:19:59.847319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:19:59.847319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:19:59.847319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:19:59.847319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:19:59.847319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:19:59.847319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:19:59.847319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:19:59.847319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:19:59.847319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:19:59.847319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:19:59.847319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:19:59.847319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:19:59.848324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:19:59.849225 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:19:59.849225 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:19:59.849225 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:19:59.849225 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 279/565] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:19:59.849976 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389abc5-9b2e-436a-adb5-63c887a102f4 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.849976 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389abc5-9b2e-436a-adb5-63c887a102f4 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.850236 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389abc5-9b2e-436a-adb5-63c887a102f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.868117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-c389abc5-9b2e-436a-adb5-63c887a102f4 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:19:59.869294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c389abc5-9b2e-436a-adb5-63c887a102f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:19:59.881509 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cc682-2bd1-4036-8ae8-e2fb711c6624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:19:59.882121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cc682-2bd1-4036-8ae8-e2fb711c6624 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:19:59.882399 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cc682-2bd1-4036-8ae8-e2fb711c6624 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:19:59.882667 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cc682-2bd1-4036-8ae8-e2fb711c6624 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:19:59.883318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cc682-2bd1-4036-8ae8-e2fb711c6624 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:19:59.883637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cc682-2bd1-4036-8ae8-e2fb711c6624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:19:59.885019 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cc682-2bd1-4036-8ae8-e2fb711c6624 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:19:59.894329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 287/566] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:19:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:19:59.895698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cc682-2bd1-4036-8ae8-e2fb711c6624 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:59.955653 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 280/567] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:19:59.955653 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389abc5-9b2e-436a-adb5-63c887a102f4 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:19:59.965208 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:00.003793 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cc682-2bd1-4036-8ae8-e2fb711c6624 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.040089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7413eb8-d130-4820-938a-ce6eb2f4865a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.040718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7413eb8-d130-4820-938a-ce6eb2f4865a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.040996 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7413eb8-d130-4820-938a-ce6eb2f4865a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.041295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7413eb8-d130-4820-938a-ce6eb2f4865a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.053226 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7413eb8-d130-4820-938a-ce6eb2f4865a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:00.055088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7413eb8-d130-4820-938a-ce6eb2f4865a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:00.062157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:00.067065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389abc5-9b2e-436a-adb5-63c887a102f4 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.071564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f7413eb8-d130-4820-938a-ce6eb2f4865a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:00.071564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:00.071564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:00.071564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:00.071564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:00.071564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:00.071564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:00.071564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:00.071564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:00.071564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:00.071564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:00.071564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:00.071564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:00.071564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:00.071564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:00.071564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:00.072722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:00.074385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:00.074385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:00.074385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:00.076993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 288/568] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:19:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:00.077688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7413eb8-d130-4820-938a-ce6eb2f4865a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.109012 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4b09f-70c0-4b6f-a8f0-1ca19dc9d596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.109669 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4b09f-70c0-4b6f-a8f0-1ca19dc9d596 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.109951 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4b09f-70c0-4b6f-a8f0-1ca19dc9d596 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.110274 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4b09f-70c0-4b6f-a8f0-1ca19dc9d596 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.110919 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4b09f-70c0-4b6f-a8f0-1ca19dc9d596 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:00.111226 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4b09f-70c0-4b6f-a8f0-1ca19dc9d596 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:00.112617 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4b09f-70c0-4b6f-a8f0-1ca19dc9d596 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:00.129301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 281/569] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:00.130167 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4b09f-70c0-4b6f-a8f0-1ca19dc9d596 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.185695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7413eb8-d130-4820-938a-ce6eb2f4865a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.204997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:00.220898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e12389-0497-459d-9a4f-fe855c4bdb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.221530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e12389-0497-459d-9a4f-fe855c4bdb04 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.221814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e12389-0497-459d-9a4f-fe855c4bdb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.222087 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e12389-0497-459d-9a4f-fe855c4bdb04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.233134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e12389-0497-459d-9a4f-fe855c4bdb04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:00.234616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e12389-0497-459d-9a4f-fe855c4bdb04 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:00.237185 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4b09f-70c0-4b6f-a8f0-1ca19dc9d596 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.248711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-b7e12389-0497-459d-9a4f-fe855c4bdb04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:00.248711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:00.248711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:00.248711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:00.248711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:00.248711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:00.248711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:00.248711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:00.248711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:00.248711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:00.248711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:00.248711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:00.248711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:00.248711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:00.248711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:00.248711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:00.249805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:00.250683 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:00.250683 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:00.250683 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:00.250683 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 289/570] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:00.250914 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e12389-0497-459d-9a4f-fe855c4bdb04 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.273441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440f43e-ea5c-44b2-b09e-ec16afb16dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.274073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440f43e-ea5c-44b2-b09e-ec16afb16dcf None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.274354 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440f43e-ea5c-44b2-b09e-ec16afb16dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.274627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440f43e-ea5c-44b2-b09e-ec16afb16dcf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.285757 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0440f43e-ea5c-44b2-b09e-ec16afb16dcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:00.290922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-0440f43e-ea5c-44b2-b09e-ec16afb16dcf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:00.290922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:00.290922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:00.290922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:00.290922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:00.290922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:00.290922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:00.290922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:00.290922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:00.290922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:00.290922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:00.290922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:00.290922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:00.290922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:00.290922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:00.290922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:00.295874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:00.295874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:00.295874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:00.295874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:00.295874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:00.295874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:00.295874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:00.295874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:00.295874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:00.295874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:00.295874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:00.295874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:00.295874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:00.295874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:00.295874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:00.295874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:00.296744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:00.296744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:00.296744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:00.296744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:00.296744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:00.296744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:00.296744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:00.296744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:00.296744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:00.296744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:00.296744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:00.296744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:00.296744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 282/571] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:00.296744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440f43e-ea5c-44b2-b09e-ec16afb16dcf None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.301720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:00.352121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e12389-0497-459d-9a4f-fe855c4bdb04 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.384769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e75a56-59b4-453b-b139-b8c0514b1019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.385915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e75a56-59b4-453b-b139-b8c0514b1019 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.386204 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e75a56-59b4-453b-b139-b8c0514b1019 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.386478 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e75a56-59b4-453b-b139-b8c0514b1019 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.387114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e75a56-59b4-453b-b139-b8c0514b1019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:00.387418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e75a56-59b4-453b-b139-b8c0514b1019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:00.388851 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e75a56-59b4-453b-b139-b8c0514b1019 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:00.392563 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440f43e-ea5c-44b2-b09e-ec16afb16dcf None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.400262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 290/572] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:00.401615 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e75a56-59b4-453b-b139-b8c0514b1019 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.420962 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:00.426824 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f3ba3-cea0-404e-8d3d-d556fbfd5886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.427483 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f3ba3-cea0-404e-8d3d-d556fbfd5886 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.427807 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f3ba3-cea0-404e-8d3d-d556fbfd5886 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.428082 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f3ba3-cea0-404e-8d3d-d556fbfd5886 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.435401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f3ba3-cea0-404e-8d3d-d556fbfd5886 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:00.436884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f3ba3-cea0-404e-8d3d-d556fbfd5886 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:00.448450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-f73f3ba3-cea0-404e-8d3d-d556fbfd5886 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:00.448450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:00.448450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:00.448450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:00.448450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:00.448450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:00.448450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:00.448450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:00.448450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:00.448450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:00.448450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:00.448450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:00.448450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:00.448450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:00.448450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:00.449694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:00.451068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:00.451068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:00.451068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:00.451068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:00.451068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:00.451068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:00.451068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:00.451068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:00.451068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 283/573] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:00.451068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f3ba3-cea0-404e-8d3d-d556fbfd5886 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.505265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e75a56-59b4-453b-b139-b8c0514b1019 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.517004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:00.540813 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb3ca5-7b2a-44fe-abb7-e501c051087d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.541440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb3ca5-7b2a-44fe-abb7-e501c051087d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.541876 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb3ca5-7b2a-44fe-abb7-e501c051087d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.542221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb3ca5-7b2a-44fe-abb7-e501c051087d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.549307 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f3ba3-cea0-404e-8d3d-d556fbfd5886 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.552110 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb3ca5-7b2a-44fe-abb7-e501c051087d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:00.556949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-3ecb3ca5-7b2a-44fe-abb7-e501c051087d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:00.556949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:00.556949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:00.556949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:00.556949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:00.556949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:00.556949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:00.556949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:00.556949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:00.556949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:00.556949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:00.556949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:00.556949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:00.556949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:00.556949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:00.556949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:00.558612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:00.558612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:00.558612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:00.558612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:00.558612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:00.558612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:00.558612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:00.558612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:00.558612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:00.558612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:00.558612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:00.558612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:00.558612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:00.558612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:00.558612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:00.558612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:00.559473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:00.559473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:00.559473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:00.559473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:00.559473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:00.559473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:00.559473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:00.559473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:00.559473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:00.559473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:00.559473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:00.559473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:00.559473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 291/574] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:00.559473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb3ca5-7b2a-44fe-abb7-e501c051087d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.587934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e350e6b-a39f-430d-972b-0be92fa053d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.589081 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e350e6b-a39f-430d-972b-0be92fa053d4 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.589354 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e350e6b-a39f-430d-972b-0be92fa053d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.589661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e350e6b-a39f-430d-972b-0be92fa053d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.590261 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e350e6b-a39f-430d-972b-0be92fa053d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:00.590564 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e350e6b-a39f-430d-972b-0be92fa053d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:00.591952 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e350e6b-a39f-430d-972b-0be92fa053d4 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:00.603337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 284/575] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:00.604138 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e350e6b-a39f-430d-972b-0be92fa053d4 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.656852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:00.658350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb3ca5-7b2a-44fe-abb7-e501c051087d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.692203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483325cf-8f25-4adc-a0fd-d66362e1bc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.692812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483325cf-8f25-4adc-a0fd-d66362e1bc88 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.693099 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483325cf-8f25-4adc-a0fd-d66362e1bc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.693374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483325cf-8f25-4adc-a0fd-d66362e1bc88 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.694036 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483325cf-8f25-4adc-a0fd-d66362e1bc88 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:00.694341 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483325cf-8f25-4adc-a0fd-d66362e1bc88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:00.695728 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483325cf-8f25-4adc-a0fd-d66362e1bc88 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:00.704006 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 292/576] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:00.704628 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483325cf-8f25-4adc-a0fd-d66362e1bc88 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.705619 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e350e6b-a39f-430d-972b-0be92fa053d4 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.736335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:00.742637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bece1c-abf7-4cba-b6d0-62d8c4cfa0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.743258 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bece1c-abf7-4cba-b6d0-62d8c4cfa0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.743540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bece1c-abf7-4cba-b6d0-62d8c4cfa0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.743846 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bece1c-abf7-4cba-b6d0-62d8c4cfa0f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.750658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bece1c-abf7-4cba-b6d0-62d8c4cfa0f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:00.752127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bece1c-abf7-4cba-b6d0-62d8c4cfa0f9 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:00.764465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-57bece1c-abf7-4cba-b6d0-62d8c4cfa0f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:00.764465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:00.764465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:00.764465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:00.764465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:00.764465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:00.764465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:00.764465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:00.764465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:00.764465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:00.764465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:00.764465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:00.764465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:00.764465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:00.764465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:00.765393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:00.766359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:00.766359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:00.766359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:00.766359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:00.766359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:00.766359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:00.766359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:00.766359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:00.766359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 285/577] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:00.766873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bece1c-abf7-4cba-b6d0-62d8c4cfa0f9 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.808965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483325cf-8f25-4adc-a0fd-d66362e1bc88 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.844250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c101f-615e-41f0-b70f-d882d62bcd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.844857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c101f-615e-41f0-b70f-d882d62bcd93 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.845137 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c101f-615e-41f0-b70f-d882d62bcd93 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.845410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c101f-615e-41f0-b70f-d882d62bcd93 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.846094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c101f-615e-41f0-b70f-d882d62bcd93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:00.846404 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c101f-615e-41f0-b70f-d882d62bcd93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:00.847856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c101f-615e-41f0-b70f-d882d62bcd93 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:00.859580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 293/578] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:00.860351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c101f-615e-41f0-b70f-d882d62bcd93 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.867521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bece1c-abf7-4cba-b6d0-62d8c4cfa0f9 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.881028 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:00.906241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdfd0a-96ca-4f43-a9be-300a145a9deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.906879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdfd0a-96ca-4f43-a9be-300a145a9deb None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.907165 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdfd0a-96ca-4f43-a9be-300a145a9deb None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.907441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdfd0a-96ca-4f43-a9be-300a145a9deb None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:00.908130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcdfd0a-96ca-4f43-a9be-300a145a9deb None admin] RBAC: Adding query filter params (name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:00.908435 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcdfd0a-96ca-4f43-a9be-300a145a9deb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:00.909813 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcdfd0a-96ca-4f43-a9be-300a145a9deb None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:00.917771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 286/579] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:00.918554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdfd0a-96ca-4f43-a9be-300a145a9deb None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:00.953977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:00.962953 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c101f-615e-41f0-b70f-d882d62bcd93 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:00.997459 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd8fe3-8543-49aa-8083-31abe1e62d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:00.998094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd8fe3-8543-49aa-8083-31abe1e62d66 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:00.998380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd8fe3-8543-49aa-8083-31abe1e62d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:00.998655 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd8fe3-8543-49aa-8083-31abe1e62d66 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.005610 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd8fe3-8543-49aa-8083-31abe1e62d66 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.007044 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd8fe3-8543-49aa-8083-31abe1e62d66 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.018627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdfd0a-96ca-4f43-a9be-300a145a9deb None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.019001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-8fbd8fe3-8543-49aa-8083-31abe1e62d66 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:20:01.019001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:01.019001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:01.019001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:01.019001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:01.019001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:01.019001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:01.019001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:01.019001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:01.019001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:01.019001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:01.019001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:01.019001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:01.019001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:01.019001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:01.019001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:01.020001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:01.020993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:01.020993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:01.020993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:01.020993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:01.020993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:01.020993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:01.020993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:01.020993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:01.020993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:20:01.020993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:01.020993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 294/580] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:01.022530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd8fe3-8543-49aa-8083-31abe1e62d66 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.052898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fe2d6-c148-45fe-b652-03ffb07acfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.053521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fe2d6-c148-45fe-b652-03ffb07acfd9 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.053800 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fe2d6-c148-45fe-b652-03ffb07acfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.054084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fe2d6-c148-45fe-b652-03ffb07acfd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.064406 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fe2d6-c148-45fe-b652-03ffb07acfd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.065848 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fe2d6-c148-45fe-b652-03ffb07acfd9 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-2c3fe2d6-c148-45fe-b652-03ffb07acfd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:01.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:01.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:01.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:01.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:01.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:01.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:01.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:01.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:01.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:01.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:01.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:01.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:01.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:01.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:01.080385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:01.081335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:01.082732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:01.082732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:01.082732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:01.082732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 287/581] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:01.082732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fe2d6-c148-45fe-b652-03ffb07acfd9 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.107294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:01.126422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd8fe3-8543-49aa-8083-31abe1e62d66 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.161538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b9d38-02c3-43e0-b227-ad635e675824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.162145 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b9d38-02c3-43e0-b227-ad635e675824 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.162425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b9d38-02c3-43e0-b227-ad635e675824 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.162700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b9d38-02c3-43e0-b227-ad635e675824 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.163388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b9d38-02c3-43e0-b227-ad635e675824 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:01.163732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b9d38-02c3-43e0-b227-ad635e675824 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.165190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b9d38-02c3-43e0-b227-ad635e675824 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.177972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 295/582] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:01.178808 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b9d38-02c3-43e0-b227-ad635e675824 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.184659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fe2d6-c148-45fe-b652-03ffb07acfd9 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.194046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:01.219596 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49d769-e434-47e3-9ef3-ae1a35861a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.220229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49d769-e434-47e3-9ef3-ae1a35861a94 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.220508 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49d769-e434-47e3-9ef3-ae1a35861a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.220810 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49d769-e434-47e3-9ef3-ae1a35861a94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.221530 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49d769-e434-47e3-9ef3-ae1a35861a94 None admin] RBAC: Adding query filter params (role.id=5a12926bd458420e81e72ecf825d67ae, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=67756e31aeee48a897cfa8bd35a10119) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:01.221837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49d769-e434-47e3-9ef3-ae1a35861a94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.223413 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49d769-e434-47e3-9ef3-ae1a35861a94 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.237879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 288/583] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/role_assignments?user.id=67756e31aeee48a897cfa8bd35a10119&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=5a12926bd458420e81e72ecf825d67ae => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:01.237879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49d769-e434-47e3-9ef3-ae1a35861a94 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.289451 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b9d38-02c3-43e0-b227-ad635e675824 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.330353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edcd1c-4931-4a72-9d17-d3c40c2a6471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.331008 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edcd1c-4931-4a72-9d17-d3c40c2a6471 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.331290 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edcd1c-4931-4a72-9d17-d3c40c2a6471 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.331561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edcd1c-4931-4a72-9d17-d3c40c2a6471 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.340950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:01.349140 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edcd1c-4931-4a72-9d17-d3c40c2a6471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.351114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edcd1c-4931-4a72-9d17-d3c40c2a6471 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.357619 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49d769-e434-47e3-9ef3-ae1a35861a94 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.368816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-88edcd1c-4931-4a72-9d17-d3c40c2a6471 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:01.368816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:01.368816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:01.368816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:01.368816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:01.368816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:01.368816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:01.368816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:01.368816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:01.368816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:01.368816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:01.368816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:01.368816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:01.368816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:01.368816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:01.368816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:01.370419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:01.371897 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:01.371897 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:01.371897 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:01.371897 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 296/584] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:01.372294 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edcd1c-4931-4a72-9d17-d3c40c2a6471 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.400467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39d98e-1c4c-4d28-8886-b6935e189b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.401028 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39d98e-1c4c-4d28-8886-b6935e189b1b None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.401307 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39d98e-1c4c-4d28-8886-b6935e189b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.401594 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39d98e-1c4c-4d28-8886-b6935e189b1b None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.402420 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39d98e-1c4c-4d28-8886-b6935e189b1b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:01.402729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39d98e-1c4c-4d28-8886-b6935e189b1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.404332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39d98e-1c4c-4d28-8886-b6935e189b1b None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.421743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 289/585] 10.222.0.58 () {56 vars in 1137 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:01.427133 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39d98e-1c4c-4d28-8886-b6935e189b1b None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.454181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:01.493784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edcd1c-4931-4a72-9d17-d3c40c2a6471 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.529978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdd3d9-e045-4bd8-98a1-f59d6aeaaf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.530591 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdd3d9-e045-4bd8-98a1-f59d6aeaaf04 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.530872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdd3d9-e045-4bd8-98a1-f59d6aeaaf04 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.531205 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdd3d9-e045-4bd8-98a1-f59d6aeaaf04 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.531902 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdd3d9-e045-4bd8-98a1-f59d6aeaaf04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:01.532204 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdd3d9-e045-4bd8-98a1-f59d6aeaaf04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.533628 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdd3d9-e045-4bd8-98a1-f59d6aeaaf04 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.537876 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39d98e-1c4c-4d28-8886-b6935e189b1b None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.546650 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 297/586] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:01.547474 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdd3d9-e045-4bd8-98a1-f59d6aeaaf04 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.583722 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5628e-7d81-457f-a9ae-db083724b07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.584766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5628e-7d81-457f-a9ae-db083724b07a None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.585481 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5628e-7d81-457f-a9ae-db083724b07a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.586831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5628e-7d81-457f-a9ae-db083724b07a None admin] PATH_INFO: `/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/311dbed3ab0b44a7830a93227ff9ad33/roles/5a12926bd458420e81e72ecf825d67ae` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.605731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5628e-7d81-457f-a9ae-db083724b07a None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user_id=311dbed3ab0b44a7830a93227ff9ad33, role_id=5a12926bd458420e81e72ecf825d67ae)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.608064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5628e-7d81-457f-a9ae-db083724b07a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.612848 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:01.635954 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 290/587] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:01 2022] PUT /identity/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/311dbed3ab0b44a7830a93227ff9ad33/roles/5a12926bd458420e81e72ecf825d67ae => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:20:01.635954 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5628e-7d81-457f-a9ae-db083724b07a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.658936 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdd3d9-e045-4bd8-98a1-f59d6aeaaf04 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.686830 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:01.705590 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d331db3-8299-4c40-a6bc-0b9453c306f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.706489 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d331db3-8299-4c40-a6bc-0b9453c306f6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.706775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d331db3-8299-4c40-a6bc-0b9453c306f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.707054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d331db3-8299-4c40-a6bc-0b9453c306f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.708065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d331db3-8299-4c40-a6bc-0b9453c306f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:01.708792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d331db3-8299-4c40-a6bc-0b9453c306f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.710215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d331db3-8299-4c40-a6bc-0b9453c306f6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.733810 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 298/588] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:01.734804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d331db3-8299-4c40-a6bc-0b9453c306f6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.783555 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5628e-7d81-457f-a9ae-db083724b07a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.830242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a807b-dc7e-466e-bf1e-2230e51eb38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.830887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a807b-dc7e-466e-bf1e-2230e51eb38b None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.831168 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a807b-dc7e-466e-bf1e-2230e51eb38b None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.831441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a807b-dc7e-466e-bf1e-2230e51eb38b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.844051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a807b-dc7e-466e-bf1e-2230e51eb38b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.845507 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a807b-dc7e-466e-bf1e-2230e51eb38b None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.860630 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d331db3-8299-4c40-a6bc-0b9453c306f6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:01.861729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-a03a807b-dc7e-466e-bf1e-2230e51eb38b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:01.861729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:01.861729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:01.861729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:01.861729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:01.861729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:01.861729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:01.861729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:01.861729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:01.861729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:01.861729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:01.861729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:01.861729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:01.861729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:01.861729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:01.861729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:01.862659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:01.863551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:01.863551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:01.863551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:01.863911 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 291/589] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:01.865076 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a807b-dc7e-466e-bf1e-2230e51eb38b None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.911653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a1df7-433e-4953-8965-a621ee21eb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:01.912506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a1df7-433e-4953-8965-a621ee21eb19 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:01.913362 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a1df7-433e-4953-8965-a621ee21eb19 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:01.913792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a1df7-433e-4953-8965-a621ee21eb19 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:01.921102 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:01.927071 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a1df7-433e-4953-8965-a621ee21eb19 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:01.930572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a1df7-433e-4953-8965-a621ee21eb19 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:01.952181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-797a1df7-433e-4953-8965-a621ee21eb19 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:20:01.952181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:01.952181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:01.952181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:01.952181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:01.952181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:01.952181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:01.952181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:01.952181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:01.952181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:01.952181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:01.952181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:01.952181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:01.952181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:01.952181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:01.952181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:01.953405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:01.954739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:01.954739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:01.954739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:01.954739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:01.954739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:01.954739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:01.954739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:01.954739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:01.954739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:20:01.954739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:01.956943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 299/590] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:01.962192 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a1df7-433e-4953-8965-a621ee21eb19 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:01.990241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a807b-dc7e-466e-bf1e-2230e51eb38b None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.026743 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.028152 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a8aed-d9ce-4abd-a775-dd1a0e41444a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.028879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a8aed-d9ce-4abd-a775-dd1a0e41444a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.029232 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a8aed-d9ce-4abd-a775-dd1a0e41444a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.029733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a8aed-d9ce-4abd-a775-dd1a0e41444a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.039001 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a8aed-d9ce-4abd-a775-dd1a0e41444a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.040881 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a8aed-d9ce-4abd-a775-dd1a0e41444a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.053773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-163a8aed-d9ce-4abd-a775-dd1a0e41444a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:20:02.053773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:02.053773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:02.053773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:02.053773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:02.053773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:02.053773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:02.053773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:02.053773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:02.053773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:02.053773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:02.053773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:02.053773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:02.053773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:02.053773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:02.053773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:02.054906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:02.056154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:02.056154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:02.056154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:02.056154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:02.056154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:02.056154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:02.056154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:02.056154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:02.056154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:20:02.056154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:02.062148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 292/591] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:02.063187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a8aed-d9ce-4abd-a775-dd1a0e41444a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.106452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a1df7-433e-4953-8965-a621ee21eb19 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.146106 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a65ed-5877-4c08-81de-b698e8cdf946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.146712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a65ed-5877-4c08-81de-b698e8cdf946 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.146991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a65ed-5877-4c08-81de-b698e8cdf946 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.147258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a65ed-5877-4c08-81de-b698e8cdf946 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.147940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a65ed-5877-4c08-81de-b698e8cdf946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:02.148243 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a65ed-5877-4c08-81de-b698e8cdf946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.149596 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a65ed-5877-4c08-81de-b698e8cdf946 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.162510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 300/592] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:02.163267 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a65ed-5877-4c08-81de-b698e8cdf946 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.202233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a8aed-d9ce-4abd-a775-dd1a0e41444a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.214381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.250758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7dd8e-98b8-43c3-a45e-e08b00f3cc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.251381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7dd8e-98b8-43c3-a45e-e08b00f3cc87 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.251757 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7dd8e-98b8-43c3-a45e-e08b00f3cc87 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.252038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7dd8e-98b8-43c3-a45e-e08b00f3cc87 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.253267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7dd8e-98b8-43c3-a45e-e08b00f3cc87 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:02.253584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7dd8e-98b8-43c3-a45e-e08b00f3cc87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.255708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb7dd8e-98b8-43c3-a45e-e08b00f3cc87 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.282486 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 293/593] 10.222.0.58 () {56 vars in 1137 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:02.283058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a65ed-5877-4c08-81de-b698e8cdf946 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.283373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7dd8e-98b8-43c3-a45e-e08b00f3cc87 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.286718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.324552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209df60e-35ea-4210-a769-2b9ba37250e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.325675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209df60e-35ea-4210-a769-2b9ba37250e7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.326061 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209df60e-35ea-4210-a769-2b9ba37250e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.326503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209df60e-35ea-4210-a769-2b9ba37250e7 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.327353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209df60e-35ea-4210-a769-2b9ba37250e7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:02.327803 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209df60e-35ea-4210-a769-2b9ba37250e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.329457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209df60e-35ea-4210-a769-2b9ba37250e7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.348794 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 301/594] 10.222.0.58 () {56 vars in 1139 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:02.349835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209df60e-35ea-4210-a769-2b9ba37250e7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.435023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb7dd8e-98b8-43c3-a45e-e08b00f3cc87 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.464672 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209df60e-35ea-4210-a769-2b9ba37250e7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.473011 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04c9e9-b52d-4950-b07d-ea111e97a706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.473820 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04c9e9-b52d-4950-b07d-ea111e97a706 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.474198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04c9e9-b52d-4950-b07d-ea111e97a706 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.474613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04c9e9-b52d-4950-b07d-ea111e97a706 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.482162 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04c9e9-b52d-4950-b07d-ea111e97a706 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.483757 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04c9e9-b52d-4950-b07d-ea111e97a706 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.496782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-1e04c9e9-b52d-4950-b07d-ea111e97a706 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:02.496782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:02.496782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:02.496782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:02.496782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:02.496782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:02.496782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:02.496782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:02.496782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:02.496782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:02.496782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:02.496782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:02.496782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:02.496782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:02.496782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:02.498125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:02.500257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:02.500257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:02.500257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:02.500257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:02.500257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:02.500257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:02.500257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:02.500257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:02.503730 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 294/595] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:02.504645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04c9e9-b52d-4950-b07d-ea111e97a706 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.506083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ea305-2291-45c2-91ac-e905d86b03d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.506719 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ea305-2291-45c2-91ac-e905d86b03d8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.507001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ea305-2291-45c2-91ac-e905d86b03d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.507275 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ea305-2291-45c2-91ac-e905d86b03d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.518925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ea305-2291-45c2-91ac-e905d86b03d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.520728 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ea305-2291-45c2-91ac-e905d86b03d8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.536405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-749ea305-2291-45c2-91ac-e905d86b03d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:02.536405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:02.536405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:02.536405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:02.536405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:02.536405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:02.536405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:02.536405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:02.536405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:02.536405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:02.536405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:02.536405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:02.536405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:02.536405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:02.536405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:02.536405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:02.537392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:02.538320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:02.538320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:02.538320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:02.540463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 302/596] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:02.540819 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.542303 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ea305-2291-45c2-91ac-e905d86b03d8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.559876 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.637652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04c9e9-b52d-4950-b07d-ea111e97a706 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.659358 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ea305-2291-45c2-91ac-e905d86b03d8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.675304 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7f48c-8f58-4c6e-b656-7d82ba3030fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.675982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7f48c-8f58-4c6e-b656-7d82ba3030fe None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.676258 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7f48c-8f58-4c6e-b656-7d82ba3030fe None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.676537 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7f48c-8f58-4c6e-b656-7d82ba3030fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.687603 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7f48c-8f58-4c6e-b656-7d82ba3030fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.689006 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7f48c-8f58-4c6e-b656-7d82ba3030fe None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.703901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3f4f5-3e31-40a5-8fb8-34156c81f8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.704233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-5ca7f48c-8f58-4c6e-b656-7d82ba3030fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:02.704233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:02.704233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:02.704233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:02.704233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:02.704233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:02.704233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:02.704233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:02.704233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:02.704233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:02.704233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:02.704233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:02.704233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:02.704233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:02.704233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:02.704233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:02.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:02.706383 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:02.706383 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:02.706383 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:02.707386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3f4f5-3e31-40a5-8fb8-34156c81f8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.708166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3f4f5-3e31-40a5-8fb8-34156c81f8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.708596 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3f4f5-3e31-40a5-8fb8-34156c81f8f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.709439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3f4f5-3e31-40a5-8fb8-34156c81f8f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:02.709900 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3f4f5-3e31-40a5-8fb8-34156c81f8f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.710063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 295/597] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:02.710910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7f48c-8f58-4c6e-b656-7d82ba3030fe None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.711806 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3f4f5-3e31-40a5-8fb8-34156c81f8f0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.735268 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 303/598] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:02.736809 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3f4f5-3e31-40a5-8fb8-34156c81f8f0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.797197 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.820554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7f48c-8f58-4c6e-b656-7d82ba3030fe None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.842622 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:02.868281 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1483d-8ecd-4d01-a101-88d6aaaaffdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.869182 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1483d-8ecd-4d01-a101-88d6aaaaffdd None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.869693 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1483d-8ecd-4d01-a101-88d6aaaaffdd None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.870249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3f4f5-3e31-40a5-8fb8-34156c81f8f0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:02.870404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1483d-8ecd-4d01-a101-88d6aaaaffdd None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.871252 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1483d-8ecd-4d01-a101-88d6aaaaffdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:02.872036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1483d-8ecd-4d01-a101-88d6aaaaffdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.874545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1483d-8ecd-4d01-a101-88d6aaaaffdd None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.887793 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 296/599] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:02.888881 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1483d-8ecd-4d01-a101-88d6aaaaffdd None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:02.921019 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d57b3-2669-4327-ba10-17d0e2a037df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:02.921735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d57b3-2669-4327-ba10-17d0e2a037df None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:02.922031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d57b3-2669-4327-ba10-17d0e2a037df None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:02.922344 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d57b3-2669-4327-ba10-17d0e2a037df None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:02.922985 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d57b3-2669-4327-ba10-17d0e2a037df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:02.923284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d57b3-2669-4327-ba10-17d0e2a037df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:02.924752 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d57b3-2669-4327-ba10-17d0e2a037df None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:02.942044 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 304/600] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:02.942955 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d57b3-2669-4327-ba10-17d0e2a037df None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.014666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1483d-8ecd-4d01-a101-88d6aaaaffdd None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:03.055268 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d5d54-9bba-4bcc-a100-36fb052648ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:03.056676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d5d54-9bba-4bcc-a100-36fb052648ff None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.057552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d5d54-9bba-4bcc-a100-36fb052648ff None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.057982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d5d54-9bba-4bcc-a100-36fb052648ff None admin] PATH_INFO: `/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/dd8df2f3357d4660bc22366f9746f89c/roles/5a12926bd458420e81e72ecf825d67ae` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.069186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:03.070793 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d57b3-2669-4327-ba10-17d0e2a037df None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:03.082891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d5d54-9bba-4bcc-a100-36fb052648ff None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user_id=dd8df2f3357d4660bc22366f9746f89c, role_id=5a12926bd458420e81e72ecf825d67ae)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:03.084829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d5d54-9bba-4bcc-a100-36fb052648ff None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.105681 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 297/601] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:02 2022] PUT /identity/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/dd8df2f3357d4660bc22366f9746f89c/roles/5a12926bd458420e81e72ecf825d67ae => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:20:03.106790 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d5d54-9bba-4bcc-a100-36fb052648ff None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.125509 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7259c0-7f7a-4600-ad91-8496f67081f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:03.126547 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7259c0-7f7a-4600-ad91-8496f67081f8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.127026 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7259c0-7f7a-4600-ad91-8496f67081f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.127590 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7259c0-7f7a-4600-ad91-8496f67081f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.133083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:03.144812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7259c0-7f7a-4600-ad91-8496f67081f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:03.146520 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7259c0-7f7a-4600-ad91-8496f67081f8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.169000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-2f7259c0-7f7a-4600-ad91-8496f67081f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:03.169000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:03.169000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:03.169000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:03.169000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:03.169000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:03.169000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:03.169000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:03.169000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:03.169000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:03.169000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:03.169000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:03.169000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:03.169000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:03.169000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:03.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:03.170922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:03.170922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:03.170922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:03.170922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:03.170922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:03.170922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:03.170922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:03.170922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:03.170922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 305/602] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:03.176487 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7259c0-7f7a-4600-ad91-8496f67081f8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.296868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d5d54-9bba-4bcc-a100-36fb052648ff None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:03.335907 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea04b73-be72-499a-954c-7b8c25daa490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:03.336866 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea04b73-be72-499a-954c-7b8c25daa490 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.336986 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7259c0-7f7a-4600-ad91-8496f67081f8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:03.337428 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea04b73-be72-499a-954c-7b8c25daa490 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.337872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea04b73-be72-499a-954c-7b8c25daa490 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.346360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea04b73-be72-499a-954c-7b8c25daa490 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:03.348254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea04b73-be72-499a-954c-7b8c25daa490 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.365829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-8ea04b73-be72-499a-954c-7b8c25daa490 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:03.365829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:03.365829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:03.365829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:03.365829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:03.365829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:03.365829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:03.365829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:03.365829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:03.365829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:03.365829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:03.365829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:03.365829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:03.365829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:03.365829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:03.366767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:03.367676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:03.367676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:03.367676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:03.367676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:03.367676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:03.367676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:03.367676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:03.367676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:03.367676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 298/603] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:03.369210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4a108-ade2-4907-9652-d1f707430aec None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.370671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4a108-ade2-4907-9652-d1f707430aec None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.370975 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4a108-ade2-4907-9652-d1f707430aec None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.373608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 299/604] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:03.375262 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4a108-ade2-4907-9652-d1f707430aec None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.382476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5727e30-44bb-40da-a61f-5d16360afa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:03.383210 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5727e30-44bb-40da-a61f-5d16360afa4c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.383591 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5727e30-44bb-40da-a61f-5d16360afa4c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.383988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5727e30-44bb-40da-a61f-5d16360afa4c None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.385558 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5727e30-44bb-40da-a61f-5d16360afa4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:03.385804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5727e30-44bb-40da-a61f-5d16360afa4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:03.387854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5727e30-44bb-40da-a61f-5d16360afa4c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.406917 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 306/605] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:03.408667 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a32f3d-5565-4864-b560-f001ac1b0b7d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.409069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a32f3d-5565-4864-b560-f001ac1b0b7d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.409441 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a32f3d-5565-4864-b560-f001ac1b0b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.429896 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-19a32f3d-5565-4864-b560-f001ac1b0b7d None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:03.431101 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-19a32f3d-5565-4864-b560-f001ac1b0b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:03.445854 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:03.491032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:03.515711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 307/606] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:03.516578 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a32f3d-5565-4864-b560-f001ac1b0b7d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.527022 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4a108-ade2-4907-9652-d1f707430aec None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:03.567988 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd756f2-39ff-4894-8625-0db0d3a1aa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:03.568602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd756f2-39ff-4894-8625-0db0d3a1aa4f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.568885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd756f2-39ff-4894-8625-0db0d3a1aa4f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.569158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd756f2-39ff-4894-8625-0db0d3a1aa4f None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.569849 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd756f2-39ff-4894-8625-0db0d3a1aa4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:03.570155 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd756f2-39ff-4894-8625-0db0d3a1aa4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:03.575814 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd756f2-39ff-4894-8625-0db0d3a1aa4f None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.600085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 300/607] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:03.605439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd756f2-39ff-4894-8625-0db0d3a1aa4f None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.700305 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a32f3d-5565-4864-b560-f001ac1b0b7d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:03.736402 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375a100-b940-4814-b9a3-c402b2083d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:03.737145 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375a100-b940-4814-b9a3-c402b2083d7c None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.737324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375a100-b940-4814-b9a3-c402b2083d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.737600 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375a100-b940-4814-b9a3-c402b2083d7c None admin] PATH_INFO: `/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/a42d8e462ba542a5bf58ba052dceeb8f/roles/5a12926bd458420e81e72ecf825d67ae` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.739644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd756f2-39ff-4894-8625-0db0d3a1aa4f None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:03.749457 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:03.753678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375a100-b940-4814-b9a3-c402b2083d7c None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user_id=a42d8e462ba542a5bf58ba052dceeb8f, role_id=5a12926bd458420e81e72ecf825d67ae)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:03.755146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375a100-b940-4814-b9a3-c402b2083d7c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.779649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 308/608] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:03 2022] PUT /identity/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/a42d8e462ba542a5bf58ba052dceeb8f/roles/5a12926bd458420e81e72ecf825d67ae => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:20:03.779649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0b8e6-3450-4aef-975c-f18bd64cc4dd None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.779649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0b8e6-3450-4aef-975c-f18bd64cc4dd None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.779649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0b8e6-3450-4aef-975c-f18bd64cc4dd None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.781444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 309/609] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:03.783136 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdffa6c-a1a3-47f0-845b-7d4018146fb9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.783472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdffa6c-a1a3-47f0-845b-7d4018146fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.783769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdffa6c-a1a3-47f0-845b-7d4018146fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.790757 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d0bb1-f868-41b0-8776-fb41eb296318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:03.792467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d0bb1-f868-41b0-8776-fb41eb296318 None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:03.793155 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d0bb1-f868-41b0-8776-fb41eb296318 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:03.793615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d0bb1-f868-41b0-8776-fb41eb296318 None admin] PATH_INFO: `/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/34fcb5885325472bbe338cf05d3788f4/roles/5a12926bd458420e81e72ecf825d67ae` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:03.804442 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5fdffa6c-a1a3-47f0-845b-7d4018146fb9 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:03.805653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5fdffa6c-a1a3-47f0-845b-7d4018146fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:03.826574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d0bb1-f868-41b0-8776-fb41eb296318 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user_id=34fcb5885325472bbe338cf05d3788f4, role_id=5a12926bd458420e81e72ecf825d67ae)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:03.828405 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d0bb1-f868-41b0-8776-fb41eb296318 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:03.837142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:03.848374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 301/610] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:03 2022] PUT /identity/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/34fcb5885325472bbe338cf05d3788f4/roles/5a12926bd458420e81e72ecf825d67ae => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:20:03.850351 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d0bb1-f868-41b0-8776-fb41eb296318 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:03.930781 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 310/611] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:20:03.971373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d0bb1-f868-41b0-8776-fb41eb296318 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.009212 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f2c22-5988-4f9f-932c-27c9b5c15444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.009875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f2c22-5988-4f9f-932c-27c9b5c15444 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.010174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f2c22-5988-4f9f-932c-27c9b5c15444 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.010973 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f2c22-5988-4f9f-932c-27c9b5c15444 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:04.022988 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f2c22-5988-4f9f-932c-27c9b5c15444 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:04.028630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-334f2c22-5988-4f9f-932c-27c9b5c15444 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:04.028630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:04.028630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:04.028630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:04.028630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:04.028630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:04.028630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:04.028630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:04.028630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:04.028630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:04.028630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:04.028630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:04.028630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:04.028630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:04.028630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:04.028630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:04.029620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:04.029620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:04.029620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:04.029620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:04.029620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:04.029620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:04.029620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:04.029620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:04.029620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:04.029620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:04.029620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:04.029620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:04.029620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:04.029620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:04.029620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:04.029620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:04.030456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:04.030456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:04.030456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:04.030456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:04.030456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:04.030456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:04.030456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:04.030456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:04.030456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:04.030456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:04.030456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:04.030456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:04.030456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 302/612] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:04.037729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdffa6c-a1a3-47f0-845b-7d4018146fb9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.158057 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdffa6c-a1a3-47f0-845b-7d4018146fb9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.199235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41179faf-1fd6-4d2b-919e-aa402ca434ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.200488 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41179faf-1fd6-4d2b-919e-aa402ca434ea None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.200970 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41179faf-1fd6-4d2b-919e-aa402ca434ea None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.201465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41179faf-1fd6-4d2b-919e-aa402ca434ea None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:04.202376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41179faf-1fd6-4d2b-919e-aa402ca434ea None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:04.202896 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41179faf-1fd6-4d2b-919e-aa402ca434ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:04.204502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41179faf-1fd6-4d2b-919e-aa402ca434ea None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:04.215580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 311/613] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:04.220660 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f2c22-5988-4f9f-932c-27c9b5c15444 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.245726 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41179faf-1fd6-4d2b-919e-aa402ca434ea None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.254712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:04.278227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:04.356602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41179faf-1fd6-4d2b-919e-aa402ca434ea None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.392383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d474d2e-c3b4-4179-b931-aa0b0e2b66b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.393018 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d474d2e-c3b4-4179-b931-aa0b0e2b66b9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.393301 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d474d2e-c3b4-4179-b931-aa0b0e2b66b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.393575 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d474d2e-c3b4-4179-b931-aa0b0e2b66b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:04.404998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d474d2e-c3b4-4179-b931-aa0b0e2b66b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:04.410418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-7d474d2e-c3b4-4179-b931-aa0b0e2b66b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:04.410418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:04.410418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:04.410418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:04.410418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:04.410418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:04.410418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:04.410418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:04.410418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:04.410418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:04.410418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:04.410418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:04.410418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:04.410418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:04.410418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:04.410418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:04.411350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:04.411350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:04.411350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:04.411350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:04.411350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:04.411350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:04.411350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:04.411350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:04.411350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:04.411350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:04.411350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:04.411350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:04.411350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:04.411350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:04.411350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:04.411350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:04.412233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:04.412233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:04.412233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:04.412233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:04.412233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:04.412233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:04.412233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:04.412233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:04.412233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:04.412233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:04.412233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:04.412233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:04.412233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 312/614] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:04.419026 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d474d2e-c3b4-4179-b931-aa0b0e2b66b9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.459071 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f2c22-5988-4f9f-932c-27c9b5c15444 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.497999 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d096a-5d89-48b9-835f-f91a5190b1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.498683 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d096a-5d89-48b9-835f-f91a5190b1ce None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.498973 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d096a-5d89-48b9-835f-f91a5190b1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.499251 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d096a-5d89-48b9-835f-f91a5190b1ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:04.510285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d096a-5d89-48b9-835f-f91a5190b1ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:04.512193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d096a-5d89-48b9-835f-f91a5190b1ce None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:04.530080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-e15d096a-5d89-48b9-835f-f91a5190b1ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:04.530080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:04.530080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:04.530080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:04.530080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:04.530080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:04.530080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:04.530080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:04.530080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:04.530080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:04.530080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:04.530080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:04.530080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:04.530080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:04.530080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:04.530080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:04.531065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:04.532052 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:04.532052 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:04.532052 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:04.532052 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 303/615] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:04.540660 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d096a-5d89-48b9-835f-f91a5190b1ce None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.632018 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:04.632215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:04.663549 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d474d2e-c3b4-4179-b931-aa0b0e2b66b9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.685545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d096a-5d89-48b9-835f-f91a5190b1ce None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.727231 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0f231-93c0-4cec-acad-846533c04b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.727864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0f231-93c0-4cec-acad-846533c04b4e None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.728148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0f231-93c0-4cec-acad-846533c04b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.728423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0f231-93c0-4cec-acad-846533c04b4e None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:04.729557 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0f231-93c0-4cec-acad-846533c04b4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:04.729801 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0f231-93c0-4cec-acad-846533c04b4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:04.731172 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0f231-93c0-4cec-acad-846533c04b4e None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:04.733885 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdcad48-e066-4529-9f9c-a6c2e68bb76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.734544 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcad48-e066-4529-9f9c-a6c2e68bb76f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.734827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcad48-e066-4529-9f9c-a6c2e68bb76f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.735099 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdcad48-e066-4529-9f9c-a6c2e68bb76f None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:04.735798 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdcad48-e066-4529-9f9c-a6c2e68bb76f None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:04.736099 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdcad48-e066-4529-9f9c-a6c2e68bb76f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:04.737994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdcad48-e066-4529-9f9c-a6c2e68bb76f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:04.747476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 313/616] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:04.752487 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdcad48-e066-4529-9f9c-a6c2e68bb76f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.756201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 304/617] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:04.762528 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0f231-93c0-4cec-acad-846533c04b4e None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:04.909939 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdcad48-e066-4529-9f9c-a6c2e68bb76f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.960875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0f231-93c0-4cec-acad-846533c04b4e None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:04.982490 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e65e35-4e66-4e56-a6d6-e8f09c9a6bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:04.983136 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e65e35-4e66-4e56-a6d6-e8f09c9a6bea None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:04.983418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e65e35-4e66-4e56-a6d6-e8f09c9a6bea None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:04.983725 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e65e35-4e66-4e56-a6d6-e8f09c9a6bea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:05.006566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e65e35-4e66-4e56-a6d6-e8f09c9a6bea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:05.008058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e65e35-4e66-4e56-a6d6-e8f09c9a6bea None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:05.023151 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:05.025425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:05.033320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-c7e65e35-4e66-4e56-a6d6-e8f09c9a6bea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:05.033320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:05.033320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:05.033320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:05.033320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:05.033320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:05.033320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:05.033320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:05.033320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:05.033320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:05.033320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:05.033320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:05.033320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:05.033320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:05.033320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:05.033320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:05.034326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:05.035223 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:05.035223 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:05.035223 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:05.035393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86efcca1-1645-4514-8a97-cf76b4380815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:05.036275 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86efcca1-1645-4514-8a97-cf76b4380815 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:05.036275 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86efcca1-1645-4514-8a97-cf76b4380815 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:05.036275 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86efcca1-1645-4514-8a97-cf76b4380815 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:05.044407 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 314/618] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:05.045474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86efcca1-1645-4514-8a97-cf76b4380815 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:05.046939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86efcca1-1645-4514-8a97-cf76b4380815 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:05.049732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e65e35-4e66-4e56-a6d6-e8f09c9a6bea None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:05.070611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-86efcca1-1645-4514-8a97-cf76b4380815 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:05.070611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:05.070611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:05.070611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:05.070611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:05.070611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:05.070611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:05.070611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:05.070611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:05.070611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:05.070611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:05.070611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:05.070611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:05.070611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:05.070611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:05.070611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:05.071732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:05.073335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:05.073335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:05.073335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:05.073335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:05.073335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:05.073335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:05.073335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:05.073335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:05.073335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:05.073335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:05.077807 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 305/619] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:05.082848 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86efcca1-1645-4514-8a97-cf76b4380815 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:05.236408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e65e35-4e66-4e56-a6d6-e8f09c9a6bea None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:05.264970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86efcca1-1645-4514-8a97-cf76b4380815 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:05.298625 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd95a79-d740-4c9c-858c-5292339363c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:05.299238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd95a79-d740-4c9c-858c-5292339363c9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:05.300542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd95a79-d740-4c9c-858c-5292339363c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:05.300820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd95a79-d740-4c9c-858c-5292339363c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:05.301473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd95a79-d740-4c9c-858c-5292339363c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:05.301778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd95a79-d740-4c9c-858c-5292339363c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:05.303154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd95a79-d740-4c9c-858c-5292339363c9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:05.331277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 315/620] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:05.333627 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656defc-4ad1-409b-8918-0d1f6c6f8b41 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:05.333912 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656defc-4ad1-409b-8918-0d1f6c6f8b41 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:05.334186 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656defc-4ad1-409b-8918-0d1f6c6f8b41 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:05.335954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 316/621] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:05.336715 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656defc-4ad1-409b-8918-0d1f6c6f8b41 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:05.341183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e77b6b-31a0-463d-98ae-8d5f03990cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:05.341805 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77b6b-31a0-463d-98ae-8d5f03990cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:05.342089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77b6b-31a0-463d-98ae-8d5f03990cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:05.342365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77b6b-31a0-463d-98ae-8d5f03990cf3 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:05.343130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e77b6b-31a0-463d-98ae-8d5f03990cf3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:05.343418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e77b6b-31a0-463d-98ae-8d5f03990cf3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:05.345009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e77b6b-31a0-463d-98ae-8d5f03990cf3 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:05.368160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 306/622] 10.222.0.58 () {56 vars in 1135 bytes} [Wed Aug 24 12:20:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:05.368160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ea504-09f2-4d7e-9d5f-6457f9fc0673 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:05.368160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ea504-09f2-4d7e-9d5f-6457f9fc0673 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:05.368160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ea504-09f2-4d7e-9d5f-6457f9fc0673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:05.383702 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-b99ea504-09f2-4d7e-9d5f-6457f9fc0673 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:05.384898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b99ea504-09f2-4d7e-9d5f-6457f9fc0673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:05.438030 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:05.441393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:05.475651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 307/623] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:05.475651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ea504-09f2-4d7e-9d5f-6457f9fc0673 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:05.511352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656defc-4ad1-409b-8918-0d1f6c6f8b41 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:05.582687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b13cf-0e26-4e2d-8b71-7ff3bfbca7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:05.583329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b13cf-0e26-4e2d-8b71-7ff3bfbca7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:05.583608 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b13cf-0e26-4e2d-8b71-7ff3bfbca7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:05.583915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b13cf-0e26-4e2d-8b71-7ff3bfbca7f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:05.603019 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b13cf-0e26-4e2d-8b71-7ff3bfbca7f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:05.604538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b13cf-0e26-4e2d-8b71-7ff3bfbca7f4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:05.621925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-4b1b13cf-0e26-4e2d-8b71-7ff3bfbca7f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:05.621925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:05.621925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:05.621925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:05.621925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:05.621925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:05.621925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:05.621925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:05.621925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:05.621925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:05.621925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:05.621925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:05.621925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:05.621925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:05.621925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:05.621925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:05.622934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:05.623925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:05.623925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:05.623925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:05.623925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:05.623925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:05.623925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:05.623925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:05.623925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:05.623925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:05.623925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:05.623925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 317/624] 10.222.0.58 () {56 vars in 1093 bytes} [Wed Aug 24 12:20:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:05.629887 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b13cf-0e26-4e2d-8b71-7ff3bfbca7f4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:05.641710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ea504-09f2-4d7e-9d5f-6457f9fc0673 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:05.681043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6904a4d-09ec-4c9f-b1c4-0462adb633d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:05.681826 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6904a4d-09ec-4c9f-b1c4-0462adb633d9 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:05.682792 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6904a4d-09ec-4c9f-b1c4-0462adb633d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:05.683161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6904a4d-09ec-4c9f-b1c4-0462adb633d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:05.704139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6904a4d-09ec-4c9f-b1c4-0462adb633d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:05.705618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6904a4d-09ec-4c9f-b1c4-0462adb633d9 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:05.731393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-b6904a4d-09ec-4c9f-b1c4-0462adb633d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:05.731393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:05.731393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:05.731393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:05.731393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:05.731393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:05.731393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:05.731393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:05.731393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:05.731393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:05.731393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:05.731393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:05.731393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:05.731393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:05.731393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:05.731393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:05.732447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:05.733351 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:05.733351 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:05.733351 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:05.735127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 308/625] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:05.739952 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6904a4d-09ec-4c9f-b1c4-0462adb633d9 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:05.787741 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b13cf-0e26-4e2d-8b71-7ff3bfbca7f4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:05.789763 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:05.824414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:05.843815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e6a41-0524-489e-9685-f0d43a9d3aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:05.844926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e6a41-0524-489e-9685-f0d43a9d3aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:05.845260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e6a41-0524-489e-9685-f0d43a9d3aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:05.845811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e6a41-0524-489e-9685-f0d43a9d3aa8 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:05.846556 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e6a41-0524-489e-9685-f0d43a9d3aa8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:05.846865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e6a41-0524-489e-9685-f0d43a9d3aa8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:05.849272 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e6a41-0524-489e-9685-f0d43a9d3aa8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:05.871452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 318/626] 10.222.0.58 () {56 vars in 1133 bytes} [Wed Aug 24 12:20:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:05.872496 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e6a41-0524-489e-9685-f0d43a9d3aa8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:05.893587 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6904a4d-09ec-4c9f-b1c4-0462adb633d9 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:05.940690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e52ea-ec18-4b00-a567-f4d052866d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:05.941341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e52ea-ec18-4b00-a567-f4d052866d9b None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:05.941625 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e52ea-ec18-4b00-a567-f4d052866d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:05.941900 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e52ea-ec18-4b00-a567-f4d052866d9b None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:05.942552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e52ea-ec18-4b00-a567-f4d052866d9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:05.942856 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e52ea-ec18-4b00-a567-f4d052866d9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:05.944264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e52ea-ec18-4b00-a567-f4d052866d9b None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:05.962237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 309/627] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:05.963110 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e52ea-ec18-4b00-a567-f4d052866d9b None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:06.014732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e6a41-0524-489e-9685-f0d43a9d3aa8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:06.061965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cc523-e204-4143-a480-da1dd6ff5087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:06.062640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cc523-e204-4143-a480-da1dd6ff5087 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.062927 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cc523-e204-4143-a480-da1dd6ff5087 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.063207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cc523-e204-4143-a480-da1dd6ff5087 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.077850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cc523-e204-4143-a480-da1dd6ff5087 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:06.081899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:06.085354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-585cc523-e204-4143-a480-da1dd6ff5087 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:06.085354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:06.085354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:06.085354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:06.085354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:06.085354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:06.085354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:06.085354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:06.085354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:06.085354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:06.085354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:06.085354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:06.085354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:06.085354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:06.085354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:06.085354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:06.086320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:06.086320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:06.086320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:06.086320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:06.086320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:06.086320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:06.086320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:06.086320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:06.086320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:06.086320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:06.086320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:06.086320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:06.086320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:06.086320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:06.086320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:06.086320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:06.087174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:06.087174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:06.087174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:06.087174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:06.087174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:06.087174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:06.087174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:06.087174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:06.087174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:06.087174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:06.087174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:06.087174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:06.091700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 319/628] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:06.092796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cc523-e204-4143-a480-da1dd6ff5087 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:06.100459 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e52ea-ec18-4b00-a567-f4d052866d9b None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:06.141405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:06.151306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f85851-8907-4b28-a24a-66121a41b25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:06.151961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f85851-8907-4b28-a24a-66121a41b25c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.152255 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f85851-8907-4b28-a24a-66121a41b25c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.152531 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f85851-8907-4b28-a24a-66121a41b25c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.210044 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cc523-e204-4143-a480-da1dd6ff5087 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:06.224089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f85851-8907-4b28-a24a-66121a41b25c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:06.225636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f85851-8907-4b28-a24a-66121a41b25c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:06.245340 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349921e7-92ae-4687-8659-ac2b89aa05fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:06.246024 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349921e7-92ae-4687-8659-ac2b89aa05fa None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.246492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349921e7-92ae-4687-8659-ac2b89aa05fa None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.246769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349921e7-92ae-4687-8659-ac2b89aa05fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.253402 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349921e7-92ae-4687-8659-ac2b89aa05fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:06.254839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349921e7-92ae-4687-8659-ac2b89aa05fa None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:06.265909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-349921e7-92ae-4687-8659-ac2b89aa05fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:06.265909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:06.265909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:06.265909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:06.265909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:06.265909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:06.265909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:06.265909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:06.265909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:06.265909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:06.265909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:06.265909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:06.265909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:06.265909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:06.265909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:06.266858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:06.267768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:06.267768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:06.267768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:06.267768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:06.267768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:06.267768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:06.267768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:06.267768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:06.267768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 320/629] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:06.267768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349921e7-92ae-4687-8659-ac2b89aa05fa None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:06.320560 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:06.331146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 310/630] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:06.332474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b22ca-e914-4e51-a9cd-5845696708cd None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.332757 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b22ca-e914-4e51-a9cd-5845696708cd None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.333026 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b22ca-e914-4e51-a9cd-5845696708cd None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.334296 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 311/631] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:06.335287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319550bb-f81c-4c68-9ed5-d494dbab99dd None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.335812 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319550bb-f81c-4c68-9ed5-d494dbab99dd None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.336097 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319550bb-f81c-4c68-9ed5-d494dbab99dd None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.337279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 312/632] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:06.337946 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319550bb-f81c-4c68-9ed5-d494dbab99dd None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:06.371112 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:06.376106 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349921e7-92ae-4687-8659-ac2b89aa05fa None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:06.411688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6c3a5-8f64-43ae-9340-cfbdc5fc3696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:06.412291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6c3a5-8f64-43ae-9340-cfbdc5fc3696 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.412571 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6c3a5-8f64-43ae-9340-cfbdc5fc3696 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.412844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6c3a5-8f64-43ae-9340-cfbdc5fc3696 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.413503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6c3a5-8f64-43ae-9340-cfbdc5fc3696 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:06.413805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6c3a5-8f64-43ae-9340-cfbdc5fc3696 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:06.415166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6c3a5-8f64-43ae-9340-cfbdc5fc3696 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:06.423726 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 321/633] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:06.424515 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6c3a5-8f64-43ae-9340-cfbdc5fc3696 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:06.439753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319550bb-f81c-4c68-9ed5-d494dbab99dd None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:06.474350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8222c-e6e7-45fb-b6e7-a269d7317be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:06.474941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8222c-e6e7-45fb-b6e7-a269d7317be7 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.475220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8222c-e6e7-45fb-b6e7-a269d7317be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.475484 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8222c-e6e7-45fb-b6e7-a269d7317be7 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.476203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8222c-e6e7-45fb-b6e7-a269d7317be7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:06.476733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8222c-e6e7-45fb-b6e7-a269d7317be7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:06.477911 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8222c-e6e7-45fb-b6e7-a269d7317be7 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:06.489797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 313/634] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:06.490951 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004f140-23d4-4fc9-bd42-df00390a4a50 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.491237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004f140-23d4-4fc9-bd42-df00390a4a50 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.491521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004f140-23d4-4fc9-bd42-df00390a4a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.508730 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-4004f140-23d4-4fc9-bd42-df00390a4a50 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:06.509938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-4004f140-23d4-4fc9-bd42-df00390a4a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:06.524384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6c3a5-8f64-43ae-9340-cfbdc5fc3696 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:06.556319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:06.558933 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711abe9-4b9e-42e9-b890-5bf0a8516124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:06.559698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711abe9-4b9e-42e9-b890-5bf0a8516124 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.559982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711abe9-4b9e-42e9-b890-5bf0a8516124 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.560259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711abe9-4b9e-42e9-b890-5bf0a8516124 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.570705 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f711abe9-4b9e-42e9-b890-5bf0a8516124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:06.572132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f711abe9-4b9e-42e9-b890-5bf0a8516124 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:06.577372 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 314/635] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:06.578603 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bd8ff-8eb9-47c6-a7dc-320acfeef98f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.578883 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bd8ff-8eb9-47c6-a7dc-320acfeef98f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.579148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bd8ff-8eb9-47c6-a7dc-320acfeef98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.594218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f711abe9-4b9e-42e9-b890-5bf0a8516124 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:06.594218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:06.594218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:06.594218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:06.594218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:06.594218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:06.594218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:06.594218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:06.594218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:06.594218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:06.594218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:06.594218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:06.594218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:06.594218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:06.594218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:06.594218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:06.595207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:06.596154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:06.596154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:06.596154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:06.596154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 322/636] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:06.596380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711abe9-4b9e-42e9-b890-5bf0a8516124 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:06.597810 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-0c0bd8ff-8eb9-47c6-a7dc-320acfeef98f None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:06.598998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-0c0bd8ff-8eb9-47c6-a7dc-320acfeef98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:06.612750 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:06.679650 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 315/637] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:06.679650 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bd8ff-8eb9-47c6-a7dc-320acfeef98f None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:06.721597 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711abe9-4b9e-42e9-b890-5bf0a8516124 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:06.767493 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62d258-6123-426c-bb28-9ea73d6332a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:06.768227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62d258-6123-426c-bb28-9ea73d6332a0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.768572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62d258-6123-426c-bb28-9ea73d6332a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.769088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd62d258-6123-426c-bb28-9ea73d6332a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.784480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62d258-6123-426c-bb28-9ea73d6332a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:06.786211 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd62d258-6123-426c-bb28-9ea73d6332a0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:06.789329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bd8ff-8eb9-47c6-a7dc-320acfeef98f None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:06.801497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:06.806052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-cd62d258-6123-426c-bb28-9ea73d6332a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:06.806052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:06.806052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:06.806052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:06.806052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:06.806052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:06.806052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:06.806052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:06.806052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:06.806052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:06.806052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:06.806052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:06.806052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:06.806052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:06.806052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:06.806052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:06.807586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:06.808529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:06.808529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:06.808529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:06.811764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 323/638] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:06.812930 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62d258-6123-426c-bb28-9ea73d6332a0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:06.832768 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db48ec-d1e4-484b-9058-a72f5fac69c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:06.833426 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db48ec-d1e4-484b-9058-a72f5fac69c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.833710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db48ec-d1e4-484b-9058-a72f5fac69c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.834126 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db48ec-d1e4-484b-9058-a72f5fac69c9 None admin] PATH_INFO: `/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/67756e31aeee48a897cfa8bd35a10119/roles/5a12926bd458420e81e72ecf825d67ae` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.850812 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db48ec-d1e4-484b-9058-a72f5fac69c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user_id=67756e31aeee48a897cfa8bd35a10119, role_id=5a12926bd458420e81e72ecf825d67ae)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:06.852381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db48ec-d1e4-484b-9058-a72f5fac69c9 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:06.865535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 316/639] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:06 2022] PUT /identity/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/67756e31aeee48a897cfa8bd35a10119/roles/5a12926bd458420e81e72ecf825d67ae => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:20:06.866412 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db48ec-d1e4-484b-9058-a72f5fac69c9 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:06.892227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:06.939272 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd62d258-6123-426c-bb28-9ea73d6332a0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:06.981570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a67b2-e591-493a-be76-65a4f737976b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:06.982259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a67b2-e591-493a-be76-65a4f737976b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:06.982524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a67b2-e591-493a-be76-65a4f737976b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:06.982855 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a67b2-e591-493a-be76-65a4f737976b None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:06.983526 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a67b2-e591-493a-be76-65a4f737976b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:06.983865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a67b2-e591-493a-be76-65a4f737976b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:06.985296 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a67b2-e591-493a-be76-65a4f737976b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:07.003498 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 324/640] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:07.003498 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a67b2-e591-493a-be76-65a4f737976b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.016742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db48ec-d1e4-484b-9058-a72f5fac69c9 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:07.059992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49812b-73f1-4ca0-8cb1-d49e8fcb0b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:07.060658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49812b-73f1-4ca0-8cb1-d49e8fcb0b2b None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.060946 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49812b-73f1-4ca0-8cb1-d49e8fcb0b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.061221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49812b-73f1-4ca0-8cb1-d49e8fcb0b2b None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.061876 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49812b-73f1-4ca0-8cb1-d49e8fcb0b2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:07.062179 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49812b-73f1-4ca0-8cb1-d49e8fcb0b2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:07.063549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49812b-73f1-4ca0-8cb1-d49e8fcb0b2b None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:07.075383 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 317/641] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:07.076331 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49812b-73f1-4ca0-8cb1-d49e8fcb0b2b None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.099461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:07.129847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a67b2-e591-493a-be76-65a4f737976b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:07.154657 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:07.166087 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d099267-d60a-4fb5-9de8-cd91fc1dd3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:07.166825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d099267-d60a-4fb5-9de8-cd91fc1dd3db None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.167204 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d099267-d60a-4fb5-9de8-cd91fc1dd3db None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.167477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d099267-d60a-4fb5-9de8-cd91fc1dd3db None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.178289 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d099267-d60a-4fb5-9de8-cd91fc1dd3db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:07.184045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-4d099267-d60a-4fb5-9de8-cd91fc1dd3db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:07.184045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:07.184045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:07.184045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:07.184045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:07.184045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:07.184045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:07.184045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:07.184045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:07.184045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:07.184045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:07.184045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:07.184045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:07.184045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:07.184045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:07.184045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:07.185004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:07.185004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:07.185004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:07.185004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:07.185004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:07.185004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:07.185004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:07.185004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:07.185004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:07.185004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:07.185004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:07.185004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:07.185004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:07.185004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:07.185004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:07.185004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:07.185850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:07.185850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:07.185850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:07.185850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:07.185850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:07.185850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:07.185850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:07.185850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:07.185850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:07.185850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:07.185850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:07.185850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:07.185850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 325/642] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:07.186574 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d099267-d60a-4fb5-9de8-cd91fc1dd3db None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.193338 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49812b-73f1-4ca0-8cb1-d49e8fcb0b2b None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:07.231317 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d964450-a56a-42c7-b6ad-cb2a07f120cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:07.232272 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d964450-a56a-42c7-b6ad-cb2a07f120cc None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.232871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d964450-a56a-42c7-b6ad-cb2a07f120cc None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.233215 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d964450-a56a-42c7-b6ad-cb2a07f120cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.244198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d964450-a56a-42c7-b6ad-cb2a07f120cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:07.251183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-9d964450-a56a-42c7-b6ad-cb2a07f120cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:07.251183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:07.251183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:07.251183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:07.251183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:07.251183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:07.251183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:07.251183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:07.251183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:07.251183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:07.251183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:07.251183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:07.251183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:07.251183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:07.251183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:07.251183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:07.252250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:07.252250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:07.252250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:07.252250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:07.252250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:07.252250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:07.252250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:07.252250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:07.252250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:07.252250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:07.252250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:07.252250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:07.252250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:07.252250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:07.252250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:07.252250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:07.253125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:07.253125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:07.253125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:07.253125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:07.253125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:07.253125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:07.253125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:07.253125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:07.253125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:07.253125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:07.253125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:07.253125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:07.253125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 318/643] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:07.253836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d964450-a56a-42c7-b6ad-cb2a07f120cc None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.296719 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d099267-d60a-4fb5-9de8-cd91fc1dd3db None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:07.338267 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b5002-0d9f-4433-a1fd-e6254ca2a85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:07.338911 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b5002-0d9f-4433-a1fd-e6254ca2a85f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.339203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b5002-0d9f-4433-a1fd-e6254ca2a85f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.339479 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b5002-0d9f-4433-a1fd-e6254ca2a85f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.342400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:07.346808 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b5002-0d9f-4433-a1fd-e6254ca2a85f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:07.348298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b5002-0d9f-4433-a1fd-e6254ca2a85f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:07.359675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d89b5002-0d9f-4433-a1fd-e6254ca2a85f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:07.359675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:07.359675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:07.359675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:07.359675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:07.359675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:07.359675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:07.359675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:07.359675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:07.359675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:07.359675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:07.359675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:07.359675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:07.359675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:07.359675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:07.360577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:07.361552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:07.361552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:07.361552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:07.361552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:07.361552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:07.361552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:07.361552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:07.361552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:07.361552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 326/644] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:07.361552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b5002-0d9f-4433-a1fd-e6254ca2a85f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.369164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d964450-a56a-42c7-b6ad-cb2a07f120cc None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:07.406241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34028e8-e729-4c1d-bbbc-64ab782cd051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:07.407280 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34028e8-e729-4c1d-bbbc-64ab782cd051 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.407668 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34028e8-e729-4c1d-bbbc-64ab782cd051 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.407936 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34028e8-e729-4c1d-bbbc-64ab782cd051 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.411468 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:07.417536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34028e8-e729-4c1d-bbbc-64ab782cd051 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:07.419437 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34028e8-e729-4c1d-bbbc-64ab782cd051 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:07.434465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-b34028e8-e729-4c1d-bbbc-64ab782cd051 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:20:07.434465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:07.434465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:07.434465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:07.434465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:07.434465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:07.434465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:07.434465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:07.434465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:07.434465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:07.434465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:07.434465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:07.434465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:07.434465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:07.434465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:07.434465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:07.435505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:07.436494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:07.436494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:07.436494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:07.436494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:07.436494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:07.436494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:07.436494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:07.436494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:07.436494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 12:20:07.436494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:07.436494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 319/645] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:07.436494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34028e8-e729-4c1d-bbbc-64ab782cd051 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.481104 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b5002-0d9f-4433-a1fd-e6254ca2a85f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:07.516959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26252741-ef0d-475c-8a5d-88357aa23027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:07.517566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26252741-ef0d-475c-8a5d-88357aa23027 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.517852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26252741-ef0d-475c-8a5d-88357aa23027 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.518128 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26252741-ef0d-475c-8a5d-88357aa23027 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.518794 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26252741-ef0d-475c-8a5d-88357aa23027 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:07.519100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26252741-ef0d-475c-8a5d-88357aa23027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:07.520504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26252741-ef0d-475c-8a5d-88357aa23027 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:07.535109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 327/646] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:07.535109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26252741-ef0d-475c-8a5d-88357aa23027 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.554586 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34028e8-e729-4c1d-bbbc-64ab782cd051 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:07.590217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c0a43-a7b6-4066-aa2b-40cc13578606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:07.590828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c0a43-a7b6-4066-aa2b-40cc13578606 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.591107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c0a43-a7b6-4066-aa2b-40cc13578606 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.591386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c0a43-a7b6-4066-aa2b-40cc13578606 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.592070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c0a43-a7b6-4066-aa2b-40cc13578606 None admin] RBAC: Adding query filter params (name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:07.592373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c0a43-a7b6-4066-aa2b-40cc13578606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:07.593964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c0a43-a7b6-4066-aa2b-40cc13578606 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:07.607251 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 320/647] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:07.607251 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c0a43-a7b6-4066-aa2b-40cc13578606 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.609887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:07.645268 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26252741-ef0d-475c-8a5d-88357aa23027 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:07.652285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:07.681362 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64bfb9-1a68-43a3-9ace-0e71c8a0ab58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:07.681970 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64bfb9-1a68-43a3-9ace-0e71c8a0ab58 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.682829 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64bfb9-1a68-43a3-9ace-0e71c8a0ab58 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.683105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64bfb9-1a68-43a3-9ace-0e71c8a0ab58 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.683829 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64bfb9-1a68-43a3-9ace-0e71c8a0ab58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:07.684139 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64bfb9-1a68-43a3-9ace-0e71c8a0ab58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:07.685596 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64bfb9-1a68-43a3-9ace-0e71c8a0ab58 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:07.698143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 328/648] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:07.699002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64bfb9-1a68-43a3-9ace-0e71c8a0ab58 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.725961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c0a43-a7b6-4066-aa2b-40cc13578606 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:07.761863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0de3a8-808a-4b30-a7d8-9a279e75f5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:07.762472 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0de3a8-808a-4b30-a7d8-9a279e75f5ca None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.762831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0de3a8-808a-4b30-a7d8-9a279e75f5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.763099 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0de3a8-808a-4b30-a7d8-9a279e75f5ca None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.763776 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0de3a8-808a-4b30-a7d8-9a279e75f5ca None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:07.764088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0de3a8-808a-4b30-a7d8-9a279e75f5ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:07.765522 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0de3a8-808a-4b30-a7d8-9a279e75f5ca None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:07.783458 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 321/649] 10.222.0.58 () {56 vars in 1137 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:07.784319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0de3a8-808a-4b30-a7d8-9a279e75f5ca None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.813667 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64bfb9-1a68-43a3-9ace-0e71c8a0ab58 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:07.847839 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:07.852485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fa1dd-dda3-40b2-adc7-f1c5d086a663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:07.853131 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fa1dd-dda3-40b2-adc7-f1c5d086a663 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.853415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fa1dd-dda3-40b2-adc7-f1c5d086a663 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.853692 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354fa1dd-dda3-40b2-adc7-f1c5d086a663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.864882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fa1dd-dda3-40b2-adc7-f1c5d086a663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:07.866276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354fa1dd-dda3-40b2-adc7-f1c5d086a663 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:07.884355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-354fa1dd-dda3-40b2-adc7-f1c5d086a663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:07.884355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:07.884355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:07.884355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:07.884355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:07.884355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:07.884355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:07.884355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:07.884355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:07.884355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:07.884355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:07.884355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:07.884355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:07.884355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:07.884355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:07.884355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:07.885472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:07.886445 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:07.886445 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:07.886445 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:07.887081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 329/650] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:07.887454 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fa1dd-dda3-40b2-adc7-f1c5d086a663 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:07.900347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0de3a8-808a-4b30-a7d8-9a279e75f5ca None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:07.910835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:07.939800 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ed2cb-147f-423f-aa74-5bd8a9cc2a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:07.940467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ed2cb-147f-423f-aa74-5bd8a9cc2a85 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:07.940717 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ed2cb-147f-423f-aa74-5bd8a9cc2a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:07.940991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ed2cb-147f-423f-aa74-5bd8a9cc2a85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:07.952567 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ed2cb-147f-423f-aa74-5bd8a9cc2a85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:07.954332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ed2cb-147f-423f-aa74-5bd8a9cc2a85 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:07.970391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-319ed2cb-147f-423f-aa74-5bd8a9cc2a85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:07.970391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:07.970391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:07.970391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:07.970391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:07.970391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:07.970391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:07.970391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:07.970391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:07.970391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:07.970391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:07.970391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:07.970391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:07.970391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:07.970391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:07.970391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:07.971419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:07.972356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:07.972356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:07.972356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:07.972356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 322/651] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:07.972356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ed2cb-147f-423f-aa74-5bd8a9cc2a85 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.003697 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354fa1dd-dda3-40b2-adc7-f1c5d086a663 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.044275 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678192c-f878-4787-8572-c92aa0848bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.044888 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678192c-f878-4787-8572-c92aa0848bab None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.045160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678192c-f878-4787-8572-c92aa0848bab None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.045437 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d678192c-f878-4787-8572-c92aa0848bab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.046159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678192c-f878-4787-8572-c92aa0848bab None admin] RBAC: Adding query filter params (role.id=5a12926bd458420e81e72ecf825d67ae, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=311dbed3ab0b44a7830a93227ff9ad33) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:08.046470 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678192c-f878-4787-8572-c92aa0848bab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.047923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d678192c-f878-4787-8572-c92aa0848bab None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.062690 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 330/652] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/role_assignments?user.id=311dbed3ab0b44a7830a93227ff9ad33&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=5a12926bd458420e81e72ecf825d67ae => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:08.062690 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678192c-f878-4787-8572-c92aa0848bab None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.116416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ed2cb-147f-423f-aa74-5bd8a9cc2a85 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.137502 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:08.147212 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:08.165445 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44459f42-6531-49e0-8b7b-95ac53cfc17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.166166 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44459f42-6531-49e0-8b7b-95ac53cfc17a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.167122 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44459f42-6531-49e0-8b7b-95ac53cfc17a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.167425 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44459f42-6531-49e0-8b7b-95ac53cfc17a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.170076 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d678192c-f878-4787-8572-c92aa0848bab None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.208621 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50939869-2449-42ee-b130-d540bcb4906c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.209538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50939869-2449-42ee-b130-d540bcb4906c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.209653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50939869-2449-42ee-b130-d540bcb4906c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.209896 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50939869-2449-42ee-b130-d540bcb4906c None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.210561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50939869-2449-42ee-b130-d540bcb4906c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:08.210869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50939869-2449-42ee-b130-d540bcb4906c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.212305 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50939869-2449-42ee-b130-d540bcb4906c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.227167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 331/653] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:08.228240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50939869-2449-42ee-b130-d540bcb4906c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.255278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44459f42-6531-49e0-8b7b-95ac53cfc17a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.257634 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44459f42-6531-49e0-8b7b-95ac53cfc17a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.363177 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50939869-2449-42ee-b130-d540bcb4906c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.392210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 323/654] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:08.393532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44459f42-6531-49e0-8b7b-95ac53cfc17a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.408093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567485a-ab94-45a0-9fd6-57636e2f3a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.408804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567485a-ab94-45a0-9fd6-57636e2f3a25 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.409186 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567485a-ab94-45a0-9fd6-57636e2f3a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.409502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d567485a-ab94-45a0-9fd6-57636e2f3a25 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.410287 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d567485a-ab94-45a0-9fd6-57636e2f3a25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:08.410814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d567485a-ab94-45a0-9fd6-57636e2f3a25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.412700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d567485a-ab94-45a0-9fd6-57636e2f3a25 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.424525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:08.428968 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 332/655] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:08.434131 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567485a-ab94-45a0-9fd6-57636e2f3a25 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.437170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:08.519982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44459f42-6531-49e0-8b7b-95ac53cfc17a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.562086 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d567485a-ab94-45a0-9fd6-57636e2f3a25 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.567843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61393275-d752-478c-85f2-658f09705b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.568492 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61393275-d752-478c-85f2-658f09705b7e None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.568775 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61393275-d752-478c-85f2-658f09705b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.569048 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61393275-d752-478c-85f2-658f09705b7e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.578969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61393275-d752-478c-85f2-658f09705b7e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.581193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61393275-d752-478c-85f2-658f09705b7e None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.597884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-61393275-d752-478c-85f2-658f09705b7e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:20:08.597884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:08.597884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:08.597884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:08.597884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:08.597884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:08.597884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:08.597884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:08.597884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:08.597884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:08.597884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:08.597884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:08.597884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:08.597884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:08.597884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:08.597884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:08.598895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:08.599873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:08.599873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:08.599873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:08.599873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:08.599873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:08.599873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:08.599873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:08.599873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:08.599873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:20:08.599873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:08.599873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 324/656] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:08.600493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61393275-d752-478c-85f2-658f09705b7e None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.610058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92633adc-27c6-4531-8c1c-1442426d6a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.610988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92633adc-27c6-4531-8c1c-1442426d6a43 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.611488 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92633adc-27c6-4531-8c1c-1442426d6a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.611994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92633adc-27c6-4531-8c1c-1442426d6a43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.626280 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92633adc-27c6-4531-8c1c-1442426d6a43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.627884 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92633adc-27c6-4531-8c1c-1442426d6a43 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-92633adc-27c6-4531-8c1c-1442426d6a43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:08.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:08.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:08.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:08.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:08.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:08.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:08.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:08.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:08.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:08.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:08.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:08.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:08.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:08.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:08.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:08.652696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:08.653613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:08.653613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:08.653613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:08.653613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 333/657] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:08.654391 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92633adc-27c6-4531-8c1c-1442426d6a43 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.692796 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:08.728078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:08.733348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61393275-d752-478c-85f2-658f09705b7e None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.775322 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425bd77-0cf8-46ff-8f33-0c7bcd1be64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.776958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425bd77-0cf8-46ff-8f33-0c7bcd1be64a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.777715 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425bd77-0cf8-46ff-8f33-0c7bcd1be64a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.778141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425bd77-0cf8-46ff-8f33-0c7bcd1be64a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.787546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92633adc-27c6-4531-8c1c-1442426d6a43 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.788270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425bd77-0cf8-46ff-8f33-0c7bcd1be64a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.790056 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425bd77-0cf8-46ff-8f33-0c7bcd1be64a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.804467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-b425bd77-0cf8-46ff-8f33-0c7bcd1be64a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:20:08.804467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:08.804467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:08.804467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:08.804467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:08.804467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:08.804467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:08.804467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:08.804467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:08.804467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:08.804467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:08.804467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:08.804467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:08.804467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:08.804467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:08.804467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:08.805517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:08.806463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:08.806463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:08.806463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:08.806463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:08.806463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:08.806463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:08.806463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:08.806463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:08.806463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 12:20:08.806463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:08.806463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 325/658] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:08.806463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425bd77-0cf8-46ff-8f33-0c7bcd1be64a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.828825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1186c0-7161-4a72-abd3-8710db640752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.829439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1186c0-7161-4a72-abd3-8710db640752 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.829721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1186c0-7161-4a72-abd3-8710db640752 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.829998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1186c0-7161-4a72-abd3-8710db640752 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.830732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1186c0-7161-4a72-abd3-8710db640752 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:08.831043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1186c0-7161-4a72-abd3-8710db640752 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.832513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1186c0-7161-4a72-abd3-8710db640752 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.856637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 334/659] 10.222.0.58 () {56 vars in 1137 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:08.856637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24349399-5339-4eca-9088-05c9dae658c9 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.856637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24349399-5339-4eca-9088-05c9dae658c9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.856637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24349399-5339-4eca-9088-05c9dae658c9 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.856637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 335/660] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:08.856637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24349399-5339-4eca-9088-05c9dae658c9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.927110 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425bd77-0cf8-46ff-8f33-0c7bcd1be64a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.958644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:08.965816 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c04e3-60dc-4fd7-9115-b84a5b6d44e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:08.966450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c04e3-60dc-4fd7-9115-b84a5b6d44e6 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:08.966704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c04e3-60dc-4fd7-9115-b84a5b6d44e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:08.966975 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c04e3-60dc-4fd7-9115-b84a5b6d44e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:08.967613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c04e3-60dc-4fd7-9115-b84a5b6d44e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:08.968013 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c04e3-60dc-4fd7-9115-b84a5b6d44e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:08.969379 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c04e3-60dc-4fd7-9115-b84a5b6d44e6 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:08.975567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24349399-5339-4eca-9088-05c9dae658c9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:08.982577 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 326/661] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:08.983883 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c04e3-60dc-4fd7-9115-b84a5b6d44e6 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:08.990352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:09.012848 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffa4c0-b02f-47e9-9a5d-e7dbdc9bf046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.013461 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffa4c0-b02f-47e9-9a5d-e7dbdc9bf046 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.013743 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffa4c0-b02f-47e9-9a5d-e7dbdc9bf046 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.014015 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffa4c0-b02f-47e9-9a5d-e7dbdc9bf046 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.014742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffa4c0-b02f-47e9-9a5d-e7dbdc9bf046 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:09.015050 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffa4c0-b02f-47e9-9a5d-e7dbdc9bf046 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.016513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffa4c0-b02f-47e9-9a5d-e7dbdc9bf046 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.037173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 336/662] 10.222.0.58 () {56 vars in 1139 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:09.039263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad3092-6c35-48e5-b2ac-2f7ff7c9eb59 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.039783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad3092-6c35-48e5-b2ac-2f7ff7c9eb59 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.040448 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad3092-6c35-48e5-b2ac-2f7ff7c9eb59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.060349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f5ad3092-6c35-48e5-b2ac-2f7ff7c9eb59 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:09.061636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f5ad3092-6c35-48e5-b2ac-2f7ff7c9eb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:09.093054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c04e3-60dc-4fd7-9115-b84a5b6d44e6 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.129015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e4347-ae67-4ee5-aa89-8c9fef8c6307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.129617 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e4347-ae67-4ee5-aa89-8c9fef8c6307 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.129897 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e4347-ae67-4ee5-aa89-8c9fef8c6307 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.130169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e4347-ae67-4ee5-aa89-8c9fef8c6307 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.132474 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 337/663] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:09.133136 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad3092-6c35-48e5-b2ac-2f7ff7c9eb59 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.210093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:09.223142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e4347-ae67-4ee5-aa89-8c9fef8c6307 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.227668 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e4347-ae67-4ee5-aa89-8c9fef8c6307 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.263816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:09.268825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad3092-6c35-48e5-b2ac-2f7ff7c9eb59 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.317080 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491f866-6ae6-4997-a39d-617361721acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.318973 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491f866-6ae6-4997-a39d-617361721acf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.319417 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491f866-6ae6-4997-a39d-617361721acf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.319885 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1491f866-6ae6-4997-a39d-617361721acf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.329447 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491f866-6ae6-4997-a39d-617361721acf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.331739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1491f866-6ae6-4997-a39d-617361721acf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.350528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-1491f866-6ae6-4997-a39d-617361721acf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:09.350528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:09.350528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:09.350528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:09.350528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:09.350528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:09.350528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:09.350528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:09.350528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:09.350528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:09.350528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:09.350528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:09.350528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:09.350528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:09.350528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:09.351713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:09.352821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:09.352821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:09.352821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:09.352821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:09.352821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:09.352821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:09.352821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:09.352821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:09.353337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 327/664] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:09.354148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e4347-ae67-4ee5-aa89-8c9fef8c6307 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.355437 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 338/665] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:09.359017 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491f866-6ae6-4997-a39d-617361721acf None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.459138 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:09.470591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e4347-ae67-4ee5-aa89-8c9fef8c6307 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.479758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1491f866-6ae6-4997-a39d-617361721acf None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.507172 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bba41-5e96-48ec-b279-bff00fcb4648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.507813 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bba41-5e96-48ec-b279-bff00fcb4648 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.508096 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bba41-5e96-48ec-b279-bff00fcb4648 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.508372 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bba41-5e96-48ec-b279-bff00fcb4648 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.515790 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1c406-7944-4c67-aab1-f172c8dbc6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.516647 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1c406-7944-4c67-aab1-f172c8dbc6ec None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.517132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1c406-7944-4c67-aab1-f172c8dbc6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.517472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1c406-7944-4c67-aab1-f172c8dbc6ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.528319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1c406-7944-4c67-aab1-f172c8dbc6ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.529763 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1c406-7944-4c67-aab1-f172c8dbc6ec None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.546959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-67e1c406-7944-4c67-aab1-f172c8dbc6ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:09.546959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:09.546959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:09.546959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:09.546959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:09.546959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:09.546959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:09.546959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:09.546959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:09.546959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:09.546959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:09.546959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:09.546959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:09.546959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:09.546959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:09.546959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:09.548010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:09.548921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:09.548921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:09.548921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:09.548921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 339/666] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:09.549876 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1c406-7944-4c67-aab1-f172c8dbc6ec None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.566475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:09.587845 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454bba41-5e96-48ec-b279-bff00fcb4648 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.590618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454bba41-5e96-48ec-b279-bff00fcb4648 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.659365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1c406-7944-4c67-aab1-f172c8dbc6ec None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.687482 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:09.696980 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e772e-c80f-4315-848b-8658d20f21c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.697609 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e772e-c80f-4315-848b-8658d20f21c0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.697881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e772e-c80f-4315-848b-8658d20f21c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.698152 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e772e-c80f-4315-848b-8658d20f21c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.703567 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 328/667] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:09.703753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e772e-c80f-4315-848b-8658d20f21c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:09.703753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e772e-c80f-4315-848b-8658d20f21c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.703897 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bba41-5e96-48ec-b279-bff00fcb4648 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.703970 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e772e-c80f-4315-848b-8658d20f21c0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.714820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 340/668] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:09.715489 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e772e-c80f-4315-848b-8658d20f21c0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.792437 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:09.809019 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bba41-5e96-48ec-b279-bff00fcb4648 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.824605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e772e-c80f-4315-848b-8658d20f21c0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:09.846937 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91881aaa-ca75-4760-a121-5a3628f84b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.847513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91881aaa-ca75-4760-a121-5a3628f84b4f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.847825 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91881aaa-ca75-4760-a121-5a3628f84b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.848126 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91881aaa-ca75-4760-a121-5a3628f84b4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.858470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91881aaa-ca75-4760-a121-5a3628f84b4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:09.860268 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0f5c8-5a89-4132-a21d-0c8315720778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:09.860884 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0f5c8-5a89-4132-a21d-0c8315720778 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:09.861166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0f5c8-5a89-4132-a21d-0c8315720778 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:09.861440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0f5c8-5a89-4132-a21d-0c8315720778 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:09.862085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0f5c8-5a89-4132-a21d-0c8315720778 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:09.862390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0f5c8-5a89-4132-a21d-0c8315720778 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:09.863174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-91881aaa-ca75-4760-a121-5a3628f84b4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:09.863174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:09.863174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:09.863174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:09.863174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:09.863174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:09.863174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:09.863174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:09.863174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:09.863174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:09.863174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:09.863174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:09.863174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:09.863174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:09.863174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:09.863174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:09.864279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:09.864279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:09.864279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:09.864279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:09.864279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:09.864279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:09.864279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:09.864279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:09.864279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:09.864279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:09.864279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:09.864279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:09.864279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:09.864279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:09.864279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:09.864279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:09.865121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:09.865121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:09.865121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:09.865121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:09.865121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:09.865121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:09.865121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:09.865121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:09.865121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:09.865121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:09.865121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:09.865121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:09.865750 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0f5c8-5a89-4132-a21d-0c8315720778 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:09.869018 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 329/669] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:09.869929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91881aaa-ca75-4760-a121-5a3628f84b4f None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.890640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 341/670] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:09.890640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0f5c8-5a89-4132-a21d-0c8315720778 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:09.932064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:09.983701 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91881aaa-ca75-4760-a121-5a3628f84b4f None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.004294 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0f5c8-5a89-4132-a21d-0c8315720778 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.027098 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121f9a0-220f-445e-8c84-083f1ba09da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.027779 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121f9a0-220f-445e-8c84-083f1ba09da2 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.028064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121f9a0-220f-445e-8c84-083f1ba09da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.028330 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121f9a0-220f-445e-8c84-083f1ba09da2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.038325 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:10.040244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121f9a0-220f-445e-8c84-083f1ba09da2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.042307 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121f9a0-220f-445e-8c84-083f1ba09da2 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.044497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf98dbe-4e58-4de6-a972-8f2eb0424c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.045379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf98dbe-4e58-4de6-a972-8f2eb0424c5e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.045665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf98dbe-4e58-4de6-a972-8f2eb0424c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf98dbe-4e58-4de6-a972-8f2eb0424c5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.046667 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf98dbe-4e58-4de6-a972-8f2eb0424c5e None admin] RBAC: Adding query filter params (role.id=5a12926bd458420e81e72ecf825d67ae, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=dd8df2f3357d4660bc22366f9746f89c) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:10.046977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf98dbe-4e58-4de6-a972-8f2eb0424c5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.048426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf98dbe-4e58-4de6-a972-8f2eb0424c5e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.063662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 342/671] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/role_assignments?user.id=dd8df2f3357d4660bc22366f9746f89c&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=5a12926bd458420e81e72ecf825d67ae => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:10.063662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf98dbe-4e58-4de6-a972-8f2eb0424c5e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.063977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-e121f9a0-220f-445e-8c84-083f1ba09da2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:10.063977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:10.063977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:10.063977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:10.063977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:10.063977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:10.063977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:10.063977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:10.063977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:10.063977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:10.063977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:10.063977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:10.063977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:10.063977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:10.063977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:10.063977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:10.065145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:10.066327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:10.066327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:10.066327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:10.066327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 330/672] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:10.066327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121f9a0-220f-445e-8c84-083f1ba09da2 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.195193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121f9a0-220f-445e-8c84-083f1ba09da2 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.200210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:10.226344 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf98dbe-4e58-4de6-a972-8f2eb0424c5e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.239998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272895c8-f302-436c-b376-c44eeeab0702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.240744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272895c8-f302-436c-b376-c44eeeab0702 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.241019 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272895c8-f302-436c-b376-c44eeeab0702 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.241405 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272895c8-f302-436c-b376-c44eeeab0702 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.250295 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272895c8-f302-436c-b376-c44eeeab0702 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.251811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272895c8-f302-436c-b376-c44eeeab0702 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.264170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-272895c8-f302-436c-b376-c44eeeab0702 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:10.264170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:10.264170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:10.264170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:10.264170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:10.264170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:10.264170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:10.264170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:10.264170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:10.264170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:10.264170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:10.264170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:10.264170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:10.264170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:10.264170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:10.265271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:10.266190 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:10.266190 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:10.266190 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:10.266190 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:10.266190 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:10.266190 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:10.266190 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:10.266190 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:10.266190 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 331/673] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:10.266190 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272895c8-f302-436c-b376-c44eeeab0702 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.271561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfb727-d0c3-4b7a-90c3-5c83422ddf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.272250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfb727-d0c3-4b7a-90c3-5c83422ddf49 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.272530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfb727-d0c3-4b7a-90c3-5c83422ddf49 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.272801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfb727-d0c3-4b7a-90c3-5c83422ddf49 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.273461 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfb727-d0c3-4b7a-90c3-5c83422ddf49 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:10.273762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfb727-d0c3-4b7a-90c3-5c83422ddf49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.275136 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfb727-d0c3-4b7a-90c3-5c83422ddf49 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.287168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 343/674] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:10.292340 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e8a9d-b134-4fe1-9a33-c82974b79225 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.292626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e8a9d-b134-4fe1-9a33-c82974b79225 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.292908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e8a9d-b134-4fe1-9a33-c82974b79225 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.294689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 344/675] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:10.294971 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e8a9d-b134-4fe1-9a33-c82974b79225 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.361791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:10.380078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272895c8-f302-436c-b376-c44eeeab0702 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.408291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e8a9d-b134-4fe1-9a33-c82974b79225 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.415819 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead838c-aacc-40fc-8506-d6968de7ea77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.416554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead838c-aacc-40fc-8506-d6968de7ea77 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.416831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead838c-aacc-40fc-8506-d6968de7ea77 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.417101 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead838c-aacc-40fc-8506-d6968de7ea77 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.417795 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead838c-aacc-40fc-8506-d6968de7ea77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:10.418100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead838c-aacc-40fc-8506-d6968de7ea77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.419471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead838c-aacc-40fc-8506-d6968de7ea77 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.437370 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 332/676] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:10.437370 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead838c-aacc-40fc-8506-d6968de7ea77 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.450676 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ffdbe3-0cb7-4a71-90e9-5fc612c90af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.451346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffdbe3-0cb7-4a71-90e9-5fc612c90af2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.451648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffdbe3-0cb7-4a71-90e9-5fc612c90af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.451920 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ffdbe3-0cb7-4a71-90e9-5fc612c90af2 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.452614 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffdbe3-0cb7-4a71-90e9-5fc612c90af2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:10.452919 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffdbe3-0cb7-4a71-90e9-5fc612c90af2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.454341 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ffdbe3-0cb7-4a71-90e9-5fc612c90af2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.462834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:10.473765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 345/677] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:10.473765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5eb9d4-5482-4afe-b54a-28f710b2e4a0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.473765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5eb9d4-5482-4afe-b54a-28f710b2e4a0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.473765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5eb9d4-5482-4afe-b54a-28f710b2e4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.491964 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1c5eb9d4-5482-4afe-b54a-28f710b2e4a0 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:10.493374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1c5eb9d4-5482-4afe-b54a-28f710b2e4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:10.547885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead838c-aacc-40fc-8506-d6968de7ea77 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.565224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 346/678] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:10.566075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5eb9d4-5482-4afe-b54a-28f710b2e4a0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.598297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f16f5-f3d9-48ba-94a3-00e9e4b9a2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.599123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f16f5-f3d9-48ba-94a3-00e9e4b9a2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.599501 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f16f5-f3d9-48ba-94a3-00e9e4b9a2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.600329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f16f5-f3d9-48ba-94a3-00e9e4b9a2c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.611741 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f16f5-f3d9-48ba-94a3-00e9e4b9a2c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.613481 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f16f5-f3d9-48ba-94a3-00e9e4b9a2c8 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.628869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:10.630733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-2c2f16f5-f3d9-48ba-94a3-00e9e4b9a2c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:10.630733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:10.630733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:10.630733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:10.630733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:10.630733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:10.630733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:10.630733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:10.630733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:10.630733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:10.630733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:10.630733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:10.630733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:10.630733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:10.630733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:10.630733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:10.632680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:10.634105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:10.634105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:10.634105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:10.634549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 333/679] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:10.636429 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f16f5-f3d9-48ba-94a3-00e9e4b9a2c8 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.688754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5eb9d4-5482-4afe-b54a-28f710b2e4a0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.723125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:10.725588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a0da0-e30b-4c05-91f9-04217aa17e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.727570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a0da0-e30b-4c05-91f9-04217aa17e00 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.727570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a0da0-e30b-4c05-91f9-04217aa17e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.727570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a0da0-e30b-4c05-91f9-04217aa17e00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.736188 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a0da0-e30b-4c05-91f9-04217aa17e00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.737959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a0da0-e30b-4c05-91f9-04217aa17e00 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.754465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f16f5-f3d9-48ba-94a3-00e9e4b9a2c8 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.756064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-195a0da0-e30b-4c05-91f9-04217aa17e00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:10.756064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:10.756064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:10.756064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:10.756064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:10.756064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:10.756064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:10.756064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:10.756064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:10.756064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:10.756064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:10.756064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:10.756064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:10.756064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:10.756064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:10.756951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:10.757889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:10.757889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:10.757889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:10.757889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:10.757889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:10.757889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:10.757889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:10.757889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:10.760892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 347/680] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:10.762054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a0da0-e30b-4c05-91f9-04217aa17e00 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.794167 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2288b32-3f5a-4251-b8b3-07d1363052eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.794785 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2288b32-3f5a-4251-b8b3-07d1363052eb None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.795070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2288b32-3f5a-4251-b8b3-07d1363052eb None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.795378 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2288b32-3f5a-4251-b8b3-07d1363052eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.796280 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2288b32-3f5a-4251-b8b3-07d1363052eb None admin] RBAC: Adding query filter params (role.id=5a12926bd458420e81e72ecf825d67ae, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=a42d8e462ba542a5bf58ba052dceeb8f) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:10.796591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2288b32-3f5a-4251-b8b3-07d1363052eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.798206 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2288b32-3f5a-4251-b8b3-07d1363052eb None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.811652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 334/681] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/role_assignments?user.id=a42d8e462ba542a5bf58ba052dceeb8f&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=5a12926bd458420e81e72ecf825d67ae => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:10.811652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2288b32-3f5a-4251-b8b3-07d1363052eb None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.906724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a0da0-e30b-4c05-91f9-04217aa17e00 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.918056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:10.927197 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2288b32-3f5a-4251-b8b3-07d1363052eb None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:10.949782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f22ed-48e3-4e7f-bfb0-83f5fa36b44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.951073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f22ed-48e3-4e7f-bfb0-83f5fa36b44e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.951587 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f22ed-48e3-4e7f-bfb0-83f5fa36b44e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.952184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f22ed-48e3-4e7f-bfb0-83f5fa36b44e None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.953081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f22ed-48e3-4e7f-bfb0-83f5fa36b44e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:10.953632 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f22ed-48e3-4e7f-bfb0-83f5fa36b44e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.955342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f22ed-48e3-4e7f-bfb0-83f5fa36b44e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.968373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8fdaf-8316-4873-a3b5-bb210d875b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:10.968994 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8fdaf-8316-4873-a3b5-bb210d875b30 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:10.969277 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8fdaf-8316-4873-a3b5-bb210d875b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:10.969550 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8fdaf-8316-4873-a3b5-bb210d875b30 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:10.970253 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8fdaf-8316-4873-a3b5-bb210d875b30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:10.970631 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8fdaf-8316-4873-a3b5-bb210d875b30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:10.971093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 348/682] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:10.972276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f22ed-48e3-4e7f-bfb0-83f5fa36b44e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:10.976640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8fdaf-8316-4873-a3b5-bb210d875b30 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:10.986899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:10.991006 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 335/683] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:10.991863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8fdaf-8316-4873-a3b5-bb210d875b30 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.112574 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f22ed-48e3-4e7f-bfb0-83f5fa36b44e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:11.113862 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8fdaf-8316-4873-a3b5-bb210d875b30 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:11.151851 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26151863-e1e9-40e4-8856-1fe90a54d7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:11.152491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26151863-e1e9-40e4-8856-1fe90a54d7f0 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.152770 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26151863-e1e9-40e4-8856-1fe90a54d7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.153045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26151863-e1e9-40e4-8856-1fe90a54d7f0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.158975 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c8826-800c-439a-a181-e197716d6cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:11.160114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c8826-800c-439a-a181-e197716d6cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.160571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26151863-e1e9-40e4-8856-1fe90a54d7f0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:11.160706 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c8826-800c-439a-a181-e197716d6cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.161124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c8826-800c-439a-a181-e197716d6cd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.161999 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26151863-e1e9-40e4-8856-1fe90a54d7f0 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:11.175700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-26151863-e1e9-40e4-8856-1fe90a54d7f0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:11.175700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:11.175700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:11.175700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:11.175700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:11.175700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:11.175700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:11.175700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:11.175700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:11.175700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:11.175700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:11.175700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:11.175700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:11.175700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:11.175700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:11.175700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:11.177023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:11.178010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:11.178010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:11.178010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:11.178010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:11.178010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:11.178010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:11.178010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:11.178010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:11.178010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:11.178010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:11.178010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 336/684] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:11.178010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26151863-e1e9-40e4-8856-1fe90a54d7f0 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.178718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c8826-800c-439a-a181-e197716d6cd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:11.186339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-9f7c8826-800c-439a-a181-e197716d6cd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:11.186339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:11.186339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:11.186339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:11.186339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:11.186339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:11.186339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:11.186339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:11.186339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:11.186339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:11.186339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:11.186339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:11.186339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:11.186339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:11.186339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:11.186339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:11.187355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:11.187355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:11.187355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:11.187355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:11.187355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:11.187355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:11.187355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:11.187355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:11.187355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:11.187355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:11.187355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:11.187355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:11.187355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:11.187355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:11.187355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:11.187355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:11.188841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:11.188841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:11.188841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:11.188841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:11.188841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:11.188841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:11.188841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:11.188841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:11.188841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:11.188841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:11.188841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:11.188841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:11.188841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 349/685] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:11.190530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c8826-800c-439a-a181-e197716d6cd2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.220530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:11.260046 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:11.309403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c8826-800c-439a-a181-e197716d6cd2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:11.315929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26151863-e1e9-40e4-8856-1fe90a54d7f0 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:11.345921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ad15f-55c0-4952-aa98-65a996b47ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:11.346886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ad15f-55c0-4952-aa98-65a996b47ecc None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.347382 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ad15f-55c0-4952-aa98-65a996b47ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.347890 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729ad15f-55c0-4952-aa98-65a996b47ecc None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.348828 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ad15f-55c0-4952-aa98-65a996b47ecc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:11.349352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ad15f-55c0-4952-aa98-65a996b47ecc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:11.350384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ba3e4-28bd-459c-a592-30b487c5e7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:11.350988 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ba3e4-28bd-459c-a592-30b487c5e7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.351262 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ba3e4-28bd-459c-a592-30b487c5e7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.351527 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ba3e4-28bd-459c-a592-30b487c5e7b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.351878 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729ad15f-55c0-4952-aa98-65a996b47ecc None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:11.352270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ba3e4-28bd-459c-a592-30b487c5e7b9 None admin] RBAC: Adding query filter params (role.id=5a12926bd458420e81e72ecf825d67ae, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=34fcb5885325472bbe338cf05d3788f4) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:11.352630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ba3e4-28bd-459c-a592-30b487c5e7b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:11.354580 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ba3e4-28bd-459c-a592-30b487c5e7b9 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:11.360857 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 337/686] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:11 2022] GET /identity/v3/role_assignments?user.id=34fcb5885325472bbe338cf05d3788f4&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=5a12926bd458420e81e72ecf825d67ae => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:11.361725 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ba3e4-28bd-459c-a592-30b487c5e7b9 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.375062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 350/687] 10.222.0.58 () {56 vars in 1135 bytes} [Wed Aug 24 12:20:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:11.383665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ad15f-55c0-4952-aa98-65a996b47ecc None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.516176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ba3e4-28bd-459c-a592-30b487c5e7b9 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:11.562021 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:11.568915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:11.585668 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd879b04-5ab9-4fde-b3fb-b5c1696b2d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:11.586282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd879b04-5ab9-4fde-b3fb-b5c1696b2d83 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.586558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd879b04-5ab9-4fde-b3fb-b5c1696b2d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.586827 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd879b04-5ab9-4fde-b3fb-b5c1696b2d83 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.587543 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd879b04-5ab9-4fde-b3fb-b5c1696b2d83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:11.587950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd879b04-5ab9-4fde-b3fb-b5c1696b2d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:11.589997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd879b04-5ab9-4fde-b3fb-b5c1696b2d83 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:11.594586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729ad15f-55c0-4952-aa98-65a996b47ecc None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:11.599126 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 338/688] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:11.604333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd879b04-5ab9-4fde-b3fb-b5c1696b2d83 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.641284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23e1d6-7b8d-4dd8-8ea0-7e7c425faa02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:11.641830 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23e1d6-7b8d-4dd8-8ea0-7e7c425faa02 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.642114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23e1d6-7b8d-4dd8-8ea0-7e7c425faa02 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.642391 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23e1d6-7b8d-4dd8-8ea0-7e7c425faa02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.742388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23e1d6-7b8d-4dd8-8ea0-7e7c425faa02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:11.745383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23e1d6-7b8d-4dd8-8ea0-7e7c425faa02 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:11.752691 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd879b04-5ab9-4fde-b3fb-b5c1696b2d83 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:11.790819 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44755c50-949e-4743-a864-b9921a43428b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:11.791490 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44755c50-949e-4743-a864-b9921a43428b None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:11.791798 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44755c50-949e-4743-a864-b9921a43428b None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:11.792070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44755c50-949e-4743-a864-b9921a43428b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:11.814624 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44755c50-949e-4743-a864-b9921a43428b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:11.816102 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44755c50-949e-4743-a864-b9921a43428b None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:11.842254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-44755c50-949e-4743-a864-b9921a43428b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:11.842254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:11.842254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:11.842254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:11.842254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:11.842254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:11.842254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:11.842254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:11.842254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:11.842254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:11.842254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:11.842254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:11.842254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:11.842254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:11.842254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:11.842254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:11.843264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:11.845872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:11.845872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:11.845872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:11.845872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 339/689] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:11.855271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44755c50-949e-4743-a864-b9921a43428b None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.892106 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:11.912878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:11.923572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 351/690] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:11.927726 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23e1d6-7b8d-4dd8-8ea0-7e7c425faa02 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:11.977595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44755c50-949e-4743-a864-b9921a43428b None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:12.015128 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d263bfb-cc62-46fc-b9e3-018f15d2087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:12.015793 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d263bfb-cc62-46fc-b9e3-018f15d2087c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:12.016073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d263bfb-cc62-46fc-b9e3-018f15d2087c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:12.016343 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d263bfb-cc62-46fc-b9e3-018f15d2087c None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:12.016986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d263bfb-cc62-46fc-b9e3-018f15d2087c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:12.017284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d263bfb-cc62-46fc-b9e3-018f15d2087c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:12.018640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d263bfb-cc62-46fc-b9e3-018f15d2087c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:12.030753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 340/691] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:12.036736 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d263bfb-cc62-46fc-b9e3-018f15d2087c None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:12.073602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23e1d6-7b8d-4dd8-8ea0-7e7c425faa02 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:12.109525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0aae96-b423-4652-b0ec-09a5227b9893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:12.110171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aae96-b423-4652-b0ec-09a5227b9893 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:12.110454 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aae96-b423-4652-b0ec-09a5227b9893 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:12.110732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0aae96-b423-4652-b0ec-09a5227b9893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:12.122397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0aae96-b423-4652-b0ec-09a5227b9893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:12.123889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0aae96-b423-4652-b0ec-09a5227b9893 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:12.142764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-5f0aae96-b423-4652-b0ec-09a5227b9893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:12.142764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:12.142764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:12.142764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:12.142764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:12.142764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:12.142764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:12.142764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:12.142764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:12.142764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:12.142764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:12.142764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:12.142764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:12.142764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:12.142764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:12.142764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:12.144972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:12.147167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:12.147167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:12.147167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:12.147551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 352/692] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:12.155857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae45e46-88b6-4544-88eb-a3cf98aff520 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:12.156329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae45e46-88b6-4544-88eb-a3cf98aff520 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:12.156700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae45e46-88b6-4544-88eb-a3cf98aff520 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:12.162266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 353/693] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:12.163256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae45e46-88b6-4544-88eb-a3cf98aff520 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:12.246459 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:12.257260 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:12.263479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d263bfb-cc62-46fc-b9e3-018f15d2087c None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:12.343082 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e413e0-280e-48f8-9fa9-3be9afc1e455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:12.343903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e413e0-280e-48f8-9fa9-3be9afc1e455 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:12.344306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e413e0-280e-48f8-9fa9-3be9afc1e455 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:12.344678 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e413e0-280e-48f8-9fa9-3be9afc1e455 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:12.356443 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e413e0-280e-48f8-9fa9-3be9afc1e455 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:12.358128 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e413e0-280e-48f8-9fa9-3be9afc1e455 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:12.377699 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae45e46-88b6-4544-88eb-a3cf98aff520 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:12.387400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-92e413e0-280e-48f8-9fa9-3be9afc1e455 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:12.387400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:12.387400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:12.387400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:12.387400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:12.387400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:12.387400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:12.387400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:12.387400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:12.387400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:12.387400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:12.387400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:12.387400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:12.387400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:12.387400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:12.387400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:12.388549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:12.389497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:12.389497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:12.389497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:12.389497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:12.389497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:12.389497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:12.389497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:12.389497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:12.389497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:12.389497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:12.394048 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 341/694] 10.222.0.58 () {56 vars in 1093 bytes} [Wed Aug 24 12:20:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:12.395236 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3bef8-c425-4677-abae-de45bf6e039d None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:12.395522 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3bef8-c425-4677-abae-de45bf6e039d None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:12.395823 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3bef8-c425-4677-abae-de45bf6e039d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:12.424459 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-a1f3bef8-c425-4677-abae-de45bf6e039d None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:12.425658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a1f3bef8-c425-4677-abae-de45bf6e039d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:12.459657 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb0f6d-a521-4104-bfea-83ab24b00211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:12.461645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb0f6d-a521-4104-bfea-83ab24b00211 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:12.464700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb0f6d-a521-4104-bfea-83ab24b00211 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:12.465476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb0f6d-a521-4104-bfea-83ab24b00211 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:12.466703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb0f6d-a521-4104-bfea-83ab24b00211 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:12.468103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb0f6d-a521-4104-bfea-83ab24b00211 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:12.470364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb0f6d-a521-4104-bfea-83ab24b00211 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:12.498222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 354/695] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:12.498222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb0f6d-a521-4104-bfea-83ab24b00211 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:12.527722 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 342/696] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:12.528710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3bef8-c425-4677-abae-de45bf6e039d None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:12.694431 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:12.701040 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb0f6d-a521-4104-bfea-83ab24b00211 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:12.739721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503f7ae-57ce-40af-a898-cd03d08ff237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:12.740685 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503f7ae-57ce-40af-a898-cd03d08ff237 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:12.741167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503f7ae-57ce-40af-a898-cd03d08ff237 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:12.741640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503f7ae-57ce-40af-a898-cd03d08ff237 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:12.742721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503f7ae-57ce-40af-a898-cd03d08ff237 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:12.743235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503f7ae-57ce-40af-a898-cd03d08ff237 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:12.745126 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503f7ae-57ce-40af-a898-cd03d08ff237 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:12.764216 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 355/697] 10.222.0.58 () {56 vars in 1133 bytes} [Wed Aug 24 12:20:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:12.772764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503f7ae-57ce-40af-a898-cd03d08ff237 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:12.811018 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:12.842520 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3bef8-c425-4677-abae-de45bf6e039d None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:12.897823 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503f7ae-57ce-40af-a898-cd03d08ff237 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:12.927368 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7b809-4211-4337-932c-278f2ca02036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:12.928112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7b809-4211-4337-932c-278f2ca02036 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:12.928375 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7b809-4211-4337-932c-278f2ca02036 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:12.928654 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7b809-4211-4337-932c-278f2ca02036 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:12.936389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7b809-4211-4337-932c-278f2ca02036 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:12.937828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7b809-4211-4337-932c-278f2ca02036 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:12.938463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38b150-16dd-4640-8477-9188a52a506b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:12.939767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38b150-16dd-4640-8477-9188a52a506b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:12.940369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38b150-16dd-4640-8477-9188a52a506b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:12.940845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38b150-16dd-4640-8477-9188a52a506b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:12.949306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-f9f7b809-4211-4337-932c-278f2ca02036 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:12.949306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:12.949306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:12.949306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:12.949306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:12.949306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:12.949306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:12.949306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:12.949306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:12.949306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:12.949306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:12.949306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:12.949306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:12.949306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:12.949306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:12.950469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:12.951382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:12.951382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:12.951382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:12.951382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:12.951382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:12.951382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:12.951382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:12.951382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:12.955389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 343/698] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:12.956269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7b809-4211-4337-932c-278f2ca02036 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:12.972157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:13.020981 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38b150-16dd-4640-8477-9188a52a506b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:13.022906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38b150-16dd-4640-8477-9188a52a506b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:13.062306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7b809-4211-4337-932c-278f2ca02036 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:13.098331 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570fc65-d06a-4fde-80bb-4f31f232bcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:13.098971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570fc65-d06a-4fde-80bb-4f31f232bcec None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.100035 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570fc65-d06a-4fde-80bb-4f31f232bcec None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.100035 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570fc65-d06a-4fde-80bb-4f31f232bcec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.112072 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570fc65-d06a-4fde-80bb-4f31f232bcec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:13.113719 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570fc65-d06a-4fde-80bb-4f31f232bcec None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:13.122796 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:13.129950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-e570fc65-d06a-4fde-80bb-4f31f232bcec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:13.129950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:13.129950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:13.129950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:13.129950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:13.129950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:13.129950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:13.129950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:13.129950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:13.129950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:13.129950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:13.129950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:13.129950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:13.129950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:13.129950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:13.129950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:13.130929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:13.131896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:13.131896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:13.131896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:13.135361 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 344/699] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:13.137246 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570fc65-d06a-4fde-80bb-4f31f232bcec None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:13.139154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 356/700] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:13.143773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38b150-16dd-4640-8477-9188a52a506b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:13.202976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:13.250724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38b150-16dd-4640-8477-9188a52a506b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:13.254901 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570fc65-d06a-4fde-80bb-4f31f232bcec None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:13.288266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ba30f-4694-402e-aa2c-64d036b102d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:13.288879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ba30f-4694-402e-aa2c-64d036b102d8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.289169 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ba30f-4694-402e-aa2c-64d036b102d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.289444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ba30f-4694-402e-aa2c-64d036b102d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.290122 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ba30f-4694-402e-aa2c-64d036b102d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:13.290426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ba30f-4694-402e-aa2c-64d036b102d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:13.292338 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6598a33-f2e8-42cd-a870-a03f8cd33a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:13.292512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ba30f-4694-402e-aa2c-64d036b102d8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:13.293237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6598a33-f2e8-42cd-a870-a03f8cd33a77 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.293657 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6598a33-f2e8-42cd-a870-a03f8cd33a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.294217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6598a33-f2e8-42cd-a870-a03f8cd33a77 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.295484 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6598a33-f2e8-42cd-a870-a03f8cd33a77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:13.295961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6598a33-f2e8-42cd-a870-a03f8cd33a77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:13.297518 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6598a33-f2e8-42cd-a870-a03f8cd33a77 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:13.311662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 357/701] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:13.311662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8faa1b-3b42-45cc-99ea-322ec3d341b6 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.311662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8faa1b-3b42-45cc-99ea-322ec3d341b6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.311662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8faa1b-3b42-45cc-99ea-322ec3d341b6 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.311662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 358/702] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:13.311662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8faa1b-3b42-45cc-99ea-322ec3d341b6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:13.315958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 345/703] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:13.317227 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824d992-efdf-4ec3-8041-ee8ae91cf126 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.317522 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824d992-efdf-4ec3-8041-ee8ae91cf126 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.317796 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824d992-efdf-4ec3-8041-ee8ae91cf126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.335769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-e824d992-efdf-4ec3-8041-ee8ae91cf126 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:13.336964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e824d992-efdf-4ec3-8041-ee8ae91cf126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:13.398213 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:13.410976 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 346/704] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:13.411896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824d992-efdf-4ec3-8041-ee8ae91cf126 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:13.429381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8faa1b-3b42-45cc-99ea-322ec3d341b6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:13.447047 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:13.470859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c289bff-ff78-4fb4-aeb4-4285a191929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:13.471802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c289bff-ff78-4fb4-aeb4-4285a191929d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.472385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c289bff-ff78-4fb4-aeb4-4285a191929d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.473008 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c289bff-ff78-4fb4-aeb4-4285a191929d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.485128 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c289bff-ff78-4fb4-aeb4-4285a191929d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:13.486819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c289bff-ff78-4fb4-aeb4-4285a191929d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:13.505114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-5c289bff-ff78-4fb4-aeb4-4285a191929d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:13.505114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:13.505114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:13.505114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:13.505114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:13.505114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:13.505114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:13.505114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:13.505114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:13.505114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:13.505114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:13.505114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:13.505114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:13.505114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:13.505114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:13.505114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:13.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:13.507566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:13.507566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:13.507566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:13.507566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 359/705] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:13.507566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c289bff-ff78-4fb4-aeb4-4285a191929d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:13.538203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824d992-efdf-4ec3-8041-ee8ae91cf126 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:13.576749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2e707-c358-4a34-8d25-ca842f028e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:13.577722 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2e707-c358-4a34-8d25-ca842f028e0a None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.578288 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2e707-c358-4a34-8d25-ca842f028e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.578759 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2e707-c358-4a34-8d25-ca842f028e0a None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.579690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2e707-c358-4a34-8d25-ca842f028e0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:13.581323 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2e707-c358-4a34-8d25-ca842f028e0a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:13.612174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 347/706] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:20:13 2022] POST /identity/v3/users => generated 254 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:13.613593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2e707-c358-4a34-8d25-ca842f028e0a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:13.649598 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c289bff-ff78-4fb4-aeb4-4285a191929d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:13.673625 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:13.703681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c1ac7-3f0c-4214-ba63-9ed20a64a3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:13.704315 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c1ac7-3f0c-4214-ba63-9ed20a64a3e9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.704853 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c1ac7-3f0c-4214-ba63-9ed20a64a3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.705130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c1ac7-3f0c-4214-ba63-9ed20a64a3e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.705870 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c1ac7-3f0c-4214-ba63-9ed20a64a3e9 None admin] RBAC: Adding query filter params (role.id=5a12926bd458420e81e72ecf825d67ae, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=67756e31aeee48a897cfa8bd35a10119) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:13.706193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c1ac7-3f0c-4214-ba63-9ed20a64a3e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:13.707694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c1ac7-3f0c-4214-ba63-9ed20a64a3e9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:13.720241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 360/707] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:13 2022] GET /identity/v3/role_assignments?user.id=67756e31aeee48a897cfa8bd35a10119&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=5a12926bd458420e81e72ecf825d67ae => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:13.720241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb4491-090e-470d-8c1e-7f1088c252b8 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.720241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb4491-090e-470d-8c1e-7f1088c252b8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.720241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb4491-090e-470d-8c1e-7f1088c252b8 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.720241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 361/708] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:13.721221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb4491-090e-470d-8c1e-7f1088c252b8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:13.757289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2e707-c358-4a34-8d25-ca842f028e0a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:13.775434 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:13.807184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7881e9f-780c-4c3b-a8c8-3b49005d1941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:13.807967 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7881e9f-780c-4c3b-a8c8-3b49005d1941 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.808315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7881e9f-780c-4c3b-a8c8-3b49005d1941 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.808655 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7881e9f-780c-4c3b-a8c8-3b49005d1941 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.809390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7881e9f-780c-4c3b-a8c8-3b49005d1941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:13.809700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7881e9f-780c-4c3b-a8c8-3b49005d1941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:13.811340 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7881e9f-780c-4c3b-a8c8-3b49005d1941 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:13.828802 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 348/709] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:13.830149 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef788-29d5-412e-ba87-add1da04478f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.830495 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef788-29d5-412e-ba87-add1da04478f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.830887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef788-29d5-412e-ba87-add1da04478f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.852666 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb4491-090e-470d-8c1e-7f1088c252b8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:13.854238 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-960ef788-29d5-412e-ba87-add1da04478f None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:13.856130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-960ef788-29d5-412e-ba87-add1da04478f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:13.908279 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6af0cd-8f55-4104-bd74-602ccb40b1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:13.909497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6af0cd-8f55-4104-bd74-602ccb40b1b3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:13.909980 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6af0cd-8f55-4104-bd74-602ccb40b1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:13.910455 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6af0cd-8f55-4104-bd74-602ccb40b1b3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:13.911056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6af0cd-8f55-4104-bd74-602ccb40b1b3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:13.913680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6af0cd-8f55-4104-bd74-602ccb40b1b3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-4d6af0cd-8f55-4104-bd74-602ccb40b1b3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:13.928517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:13.929629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:13.929629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:13.929629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:13.929629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:13.929629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:13.929629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:13.929629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:13.929629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:13.929629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:13.929629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:13.929629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:13.929629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:13.929629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:13.929629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:13.929629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:13.929629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:13.930510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:13.930510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:13.930510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:13.930510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:13.930510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:13.930510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:13.930510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:13.930510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:13.930510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:13.930510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:13.930510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:13.930510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:13.930510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:13.930510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:13.930510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 362/710] 10.222.0.58 () {56 vars in 1129 bytes} [Wed Aug 24 12:20:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:13.931323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6af0cd-8f55-4104-bd74-602ccb40b1b3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:13.955798 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 349/711] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:13.956734 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ef788-29d5-412e-ba87-add1da04478f None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:13.999592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:14.065644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:14.073932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6af0cd-8f55-4104-bd74-602ccb40b1b3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:14.090311 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ef788-29d5-412e-ba87-add1da04478f None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:14.113416 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e898aac-2387-448a-9add-acaeb1a88aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:14.114149 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e898aac-2387-448a-9add-acaeb1a88aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:14.114524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e898aac-2387-448a-9add-acaeb1a88aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:14.114873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e898aac-2387-448a-9add-acaeb1a88aa7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:14.124744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e898aac-2387-448a-9add-acaeb1a88aa7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:14.126848 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e898aac-2387-448a-9add-acaeb1a88aa7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:14.133089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8c0c4-0274-4540-9a6b-db800dbcfce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:14.134231 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8c0c4-0274-4540-9a6b-db800dbcfce7 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:14.134709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8c0c4-0274-4540-9a6b-db800dbcfce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:14.135828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed8c0c4-0274-4540-9a6b-db800dbcfce7 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:14.136726 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8c0c4-0274-4540-9a6b-db800dbcfce7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:14.137204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8c0c4-0274-4540-9a6b-db800dbcfce7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:14.138758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed8c0c4-0274-4540-9a6b-db800dbcfce7 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:14.142372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-9e898aac-2387-448a-9add-acaeb1a88aa7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:14.142372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:14.142372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:14.142372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:14.142372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:14.142372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:14.142372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:14.142372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:14.142372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:14.142372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:14.142372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:14.142372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:14.142372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:14.142372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:14.142372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:14.143263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:14.144233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:14.144233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:14.144233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:14.144233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:14.144233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:14.144233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:14.144233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:14.144233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:14.144233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 363/712] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:14.147892 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 350/713] 10.222.0.58 () {56 vars in 1124 bytes} [Wed Aug 24 12:20:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:14.153239 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e898aac-2387-448a-9add-acaeb1a88aa7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:14.155298 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8c0c4-0274-4540-9a6b-db800dbcfce7 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:14.264378 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:14.267724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed8c0c4-0274-4540-9a6b-db800dbcfce7 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:14.269707 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e898aac-2387-448a-9add-acaeb1a88aa7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:14.310479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfca851-074c-4acf-8da6-543c95b46338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:14.311433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfca851-074c-4acf-8da6-543c95b46338 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:14.311895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfca851-074c-4acf-8da6-543c95b46338 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:14.312315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfca851-074c-4acf-8da6-543c95b46338 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:14.313120 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfca851-074c-4acf-8da6-543c95b46338 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:14.313575 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfca851-074c-4acf-8da6-543c95b46338 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:14.315106 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfca851-074c-4acf-8da6-543c95b46338 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:14.321784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c8120-77ea-4689-a5dd-0c699d5bdfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:14.321970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 351/714] 10.222.0.58 () {56 vars in 1124 bytes} [Wed Aug 24 12:20:14 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:14.322397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c8120-77ea-4689-a5dd-0c699d5bdfe2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:14.322742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c8120-77ea-4689-a5dd-0c699d5bdfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:14.323087 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c8120-77ea-4689-a5dd-0c699d5bdfe2 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:14.323469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfca851-074c-4acf-8da6-543c95b46338 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:14.323895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c8120-77ea-4689-a5dd-0c699d5bdfe2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:14.324208 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c8120-77ea-4689-a5dd-0c699d5bdfe2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:14.325960 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c8120-77ea-4689-a5dd-0c699d5bdfe2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:14.341239 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 364/715] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:14.346668 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c8120-77ea-4689-a5dd-0c699d5bdfe2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:14.401256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:14.499816 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfca851-074c-4acf-8da6-543c95b46338 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:14.540104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8a260-baa4-4063-8c07-ab32919e4dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:14.540833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8a260-baa4-4063-8c07-ab32919e4dbf None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:14.541120 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8a260-baa4-4063-8c07-ab32919e4dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:14.541433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8a260-baa4-4063-8c07-ab32919e4dbf None admin] PATH_INFO: `/v3/services/network` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:14.542920 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8a260-baa4-4063-8c07-ab32919e4dbf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:14.544329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc8a260-baa4-4063-8c07-ab32919e4dbf None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-5fc8a260-baa4-4063-8c07-ab32919e4dbf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:14.556427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:14.557666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:14.557666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:14.557666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:14.557666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:14.557666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:14.557666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:14.557666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:14.557666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:14.557666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:14.557666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:14.557666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:14.557666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:14.557666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:14.557666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:14.557666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:14.557666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:14.558509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:14.558509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:14.558509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:14.558509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:14.558509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:14.558509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:14.558509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:14.558509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:14.558509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:14.558509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:14.558509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:14.558509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:14.558509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:14.558509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:14.558509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 352/716] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:14 2022] GET /identity/v3/services/network => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:14.564064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8a260-baa4-4063-8c07-ab32919e4dbf None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:14.604480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c8120-77ea-4689-a5dd-0c699d5bdfe2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:14.617824 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:14.670622 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c238a-ee87-4672-979b-8d6516e7b070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:14.671576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c238a-ee87-4672-979b-8d6516e7b070 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:14.672063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c238a-ee87-4672-979b-8d6516e7b070 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:14.673211 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c238a-ee87-4672-979b-8d6516e7b070 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:14.674140 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c238a-ee87-4672-979b-8d6516e7b070 None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=311dbed3ab0b44a7830a93227ff9ad33) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:14.675548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c238a-ee87-4672-979b-8d6516e7b070 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:14.677296 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c238a-ee87-4672-979b-8d6516e7b070 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:14.685729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 365/717] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:14 2022] GET /identity/v3/role_assignments?user.id=311dbed3ab0b44a7830a93227ff9ad33&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 259 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:14.748883 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc8a260-baa4-4063-8c07-ab32919e4dbf None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:14.810932 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca7a9f5-3266-4564-b478-eef7cb5b80b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:14.811614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7a9f5-3266-4564-b478-eef7cb5b80b0 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:14.812175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7a9f5-3266-4564-b478-eef7cb5b80b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:14.812175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca7a9f5-3266-4564-b478-eef7cb5b80b0 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:14.812875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7a9f5-3266-4564-b478-eef7cb5b80b0 None admin] RBAC: Adding query filter params (name=network) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:14.813186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7a9f5-3266-4564-b478-eef7cb5b80b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:14.814604 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca7a9f5-3266-4564-b478-eef7cb5b80b0 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:14.820036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 353/718] 10.222.0.58 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:14.826045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c238a-ee87-4672-979b-8d6516e7b070 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:14.957741 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:14.984679 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c238a-ee87-4672-979b-8d6516e7b070 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:15.033506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c558526-2e06-4c80-a507-7ab0766ee626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:15.034509 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c558526-2e06-4c80-a507-7ab0766ee626 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:15.035000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c558526-2e06-4c80-a507-7ab0766ee626 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:15.035502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c558526-2e06-4c80-a507-7ab0766ee626 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:15.036394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c558526-2e06-4c80-a507-7ab0766ee626 None admin] RBAC: Adding query filter params (type=network) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:15.036949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c558526-2e06-4c80-a507-7ab0766ee626 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:15.038599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c558526-2e06-4c80-a507-7ab0766ee626 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:15.048757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 366/719] 10.222.0.58 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:14 2022] GET /identity/v3/services?type=network => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:16.069933 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af029ef-e0df-4940-b099-62f44b0bf8b8 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.070475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af029ef-e0df-4940-b099-62f44b0bf8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.070475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af029ef-e0df-4940-b099-62f44b0bf8b8 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.072125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 354/720] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:16.078324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e43f6-65fb-46bd-b913-48c95dcf7af3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.078741 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e43f6-65fb-46bd-b913-48c95dcf7af3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.078936 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e43f6-65fb-46bd-b913-48c95dcf7af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.101819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ea1e43f6-65fb-46bd-b913-48c95dcf7af3 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:16.103016 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ea1e43f6-65fb-46bd-b913-48c95dcf7af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:16.173337 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 367/721] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:16.420612 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f583c-b57a-4d0c-9926-484fa8c6ff92 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.420890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f583c-b57a-4d0c-9926-484fa8c6ff92 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.421165 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f583c-b57a-4d0c-9926-484fa8c6ff92 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.422774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 355/722] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:16.428383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55160d6-907d-460e-ba83-56d700e5806f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.428613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55160d6-907d-460e-ba83-56d700e5806f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.428887 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55160d6-907d-460e-ba83-56d700e5806f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.457649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a55160d6-907d-460e-ba83-56d700e5806f None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:16.458845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a55160d6-907d-460e-ba83-56d700e5806f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:16.542520 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 368/723] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:16.590597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f583c-b57a-4d0c-9926-484fa8c6ff92 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:16.669825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03914d1f-38ca-45b3-8c4e-3cd8596f1354 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.670178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03914d1f-38ca-45b3-8c4e-3cd8596f1354 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.670388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03914d1f-38ca-45b3-8c4e-3cd8596f1354 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.671968 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 369/724] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:16.676978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef60397-15df-487a-ade1-c084b73a86e7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.677275 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef60397-15df-487a-ade1-c084b73a86e7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.677553 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef60397-15df-487a-ade1-c084b73a86e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.704722 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:16.712063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:16.721677 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1ef60397-15df-487a-ade1-c084b73a86e7 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:16.722898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1ef60397-15df-487a-ade1-c084b73a86e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:16.724260 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f583c-b57a-4d0c-9926-484fa8c6ff92 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:16.795590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9f365-c6fd-4854-8186-f2dc18509858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:16.796277 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9f365-c6fd-4854-8186-f2dc18509858 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:16.796561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9f365-c6fd-4854-8186-f2dc18509858 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:16.796837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9f365-c6fd-4854-8186-f2dc18509858 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:16.818959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc9f365-c6fd-4854-8186-f2dc18509858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:16.824269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-bcc9f365-c6fd-4854-8186-f2dc18509858 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:16.824269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:16.824269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:16.824269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:16.824269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:16.824269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:16.824269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:16.824269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:16.824269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:16.824269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:16.824269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:16.824269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:16.824269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:16.824269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:16.824269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:16.824269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:16.825479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:16.825479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:16.825479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:16.825479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:16.825479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:16.825479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:16.825479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:16.825479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:16.825479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:16.825479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:16.825479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:16.825479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:16.825479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:16.825479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:16.825479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:16.825479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:16.826358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:16.826358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:16.826358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:16.826358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:16.826358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:16.826358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:16.826358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:16.826358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:16.826358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:16.826358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:16.826358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:16.826358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:16.831654 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 356/725] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:16.835974 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9f365-c6fd-4854-8186-f2dc18509858 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:16.864607 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 370/726] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 12:20:16.865523 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef60397-15df-487a-ade1-c084b73a86e7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:16.984877 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef60397-15df-487a-ade1-c084b73a86e7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:16.990111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9f365-c6fd-4854-8186-f2dc18509858 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.021693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779cdeb1-009a-4330-bb0f-b9b357e622ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.022756 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cdeb1-009a-4330-bb0f-b9b357e622ce None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.023181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cdeb1-009a-4330-bb0f-b9b357e622ce None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.023599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779cdeb1-009a-4330-bb0f-b9b357e622ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.025727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:17.032157 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8e35a-c029-4680-9de0-df767be68b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.032881 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8e35a-c029-4680-9de0-df767be68b60 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.033173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8e35a-c029-4680-9de0-df767be68b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.033449 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8e35a-c029-4680-9de0-df767be68b60 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.034284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8e35a-c029-4680-9de0-df767be68b60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:17.034596 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8e35a-c029-4680-9de0-df767be68b60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:17.036000 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8e35a-c029-4680-9de0-df767be68b60 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:17.037771 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779cdeb1-009a-4330-bb0f-b9b357e622ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:17.044094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-779cdeb1-009a-4330-bb0f-b9b357e622ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:17.044094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:17.044094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:17.044094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:17.044094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:17.044094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:17.044094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:17.044094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:17.044094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:17.044094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:17.044094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:17.044094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:17.044094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:17.044094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:17.044094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:17.044094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:17.045183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:17.045183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:17.045183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:17.045183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:17.045183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:17.045183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:17.045183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:17.045183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:17.045183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:17.045183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:17.045183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:17.045183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:17.045183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:17.045183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:17.045183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:17.045183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:17.046227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:17.046227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:17.046227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:17.046227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:17.046227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:17.046227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:17.046227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:17.046227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:17.046227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:17.046227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:17.046227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:17.046227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:17.046872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 357/727] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:17.046872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e578c-5c9b-4280-a1c0-d3c985a26580 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.046872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e578c-5c9b-4280-a1c0-d3c985a26580 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.046872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e578c-5c9b-4280-a1c0-d3c985a26580 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.053474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 358/729] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:17.053474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e578c-5c9b-4280-a1c0-d3c985a26580 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.053758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 371/729] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:17.056020 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db153137-db50-428e-84c6-97ec0a6d597b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.057959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db153137-db50-428e-84c6-97ec0a6d597b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.058492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db153137-db50-428e-84c6-97ec0a6d597b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.073432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:17.084590 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-db153137-db50-428e-84c6-97ec0a6d597b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:17.085287 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-db153137-db50-428e-84c6-97ec0a6d597b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:17.171720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 372/730] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:17.172879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db153137-db50-428e-84c6-97ec0a6d597b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.178033 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e578c-5c9b-4280-a1c0-d3c985a26580 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.222352 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df9ebb-afb4-4028-9cec-c117b9e11d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.223037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df9ebb-afb4-4028-9cec-c117b9e11d2a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.223321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df9ebb-afb4-4028-9cec-c117b9e11d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.223590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df9ebb-afb4-4028-9cec-c117b9e11d2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.238986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df9ebb-afb4-4028-9cec-c117b9e11d2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:17.240924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df9ebb-afb4-4028-9cec-c117b9e11d2a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:17.262316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-51df9ebb-afb4-4028-9cec-c117b9e11d2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:17.262316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:17.262316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:17.262316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:17.262316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:17.262316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:17.262316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:17.262316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:17.262316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:17.262316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:17.262316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:17.262316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:17.262316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:17.262316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:17.262316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:17.262316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:17.263332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:17.264574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:17.264574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:17.264574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:17.264574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 359/731] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:17.264574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df9ebb-afb4-4028-9cec-c117b9e11d2a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.299989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db153137-db50-428e-84c6-97ec0a6d597b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.326966 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:17.342246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e5b28-357f-4cd5-a4ff-82fb094bc225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.342870 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e5b28-357f-4cd5-a4ff-82fb094bc225 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.343152 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e5b28-357f-4cd5-a4ff-82fb094bc225 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.343428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e5b28-357f-4cd5-a4ff-82fb094bc225 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.344403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e5b28-357f-4cd5-a4ff-82fb094bc225 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:17.344711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e5b28-357f-4cd5-a4ff-82fb094bc225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:17.346260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e5b28-357f-4cd5-a4ff-82fb094bc225 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:17.353276 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:17.358811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 373/732] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:17.358811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e5b28-357f-4cd5-a4ff-82fb094bc225 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.407694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df9ebb-afb4-4028-9cec-c117b9e11d2a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.445503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158e3d8-c8a6-4b1a-ad0b-cd6e63fa7f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.446576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158e3d8-c8a6-4b1a-ad0b-cd6e63fa7f88 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.447060 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158e3d8-c8a6-4b1a-ad0b-cd6e63fa7f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.448181 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158e3d8-c8a6-4b1a-ad0b-cd6e63fa7f88 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.449032 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158e3d8-c8a6-4b1a-ad0b-cd6e63fa7f88 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:17.450933 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158e3d8-c8a6-4b1a-ad0b-cd6e63fa7f88 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:17.473479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 360/733] 10.222.0.58 () {60 vars in 1141 bytes} [Wed Aug 24 12:20:17 2022] POST /identity/v3/services => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:17.474718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158e3d8-c8a6-4b1a-ad0b-cd6e63fa7f88 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.494499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e5b28-357f-4cd5-a4ff-82fb094bc225 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.554240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016e4f0-ffa1-4707-a60a-a1a04c513fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.554867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016e4f0-ffa1-4707-a60a-a1a04c513fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.555147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016e4f0-ffa1-4707-a60a-a1a04c513fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.555419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016e4f0-ffa1-4707-a60a-a1a04c513fc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.556109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016e4f0-ffa1-4707-a60a-a1a04c513fc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:17.556412 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016e4f0-ffa1-4707-a60a-a1a04c513fc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:17.557965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016e4f0-ffa1-4707-a60a-a1a04c513fc1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:17.579730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 374/734] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:17.580826 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016e4f0-ffa1-4707-a60a-a1a04c513fc1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.621418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158e3d8-c8a6-4b1a-ad0b-cd6e63fa7f88 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.640345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:17.657576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:17.670203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239de6aa-5588-4c6c-a873-85df3f0b2702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.670863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239de6aa-5588-4c6c-a873-85df3f0b2702 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.671145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239de6aa-5588-4c6c-a873-85df3f0b2702 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.671570 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239de6aa-5588-4c6c-a873-85df3f0b2702 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.685860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239de6aa-5588-4c6c-a873-85df3f0b2702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:17.689707 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239de6aa-5588-4c6c-a873-85df3f0b2702 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:17.708593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-239de6aa-5588-4c6c-a873-85df3f0b2702 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:17.708593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:17.708593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:17.708593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:17.708593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:17.708593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:17.708593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:17.708593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:17.708593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:17.708593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:17.708593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:17.708593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:17.708593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:17.708593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:17.708593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:17.708593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:17.709930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:17.710838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:17.710838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:17.710838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:17.710838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 361/735] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:17.710838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239de6aa-5588-4c6c-a873-85df3f0b2702 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.725780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016e4f0-ffa1-4707-a60a-a1a04c513fc1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.762514 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c9372-9f51-4f06-ac6a-b1c0b236d349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.763068 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c9372-9f51-4f06-ac6a-b1c0b236d349 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.763348 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c9372-9f51-4f06-ac6a-b1c0b236d349 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.763643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c9372-9f51-4f06-ac6a-b1c0b236d349 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.773809 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c9372-9f51-4f06-ac6a-b1c0b236d349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:17.780001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-c27c9372-9f51-4f06-ac6a-b1c0b236d349 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:17.780001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:17.780001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:17.780001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:17.780001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:17.780001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:17.780001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:17.780001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:17.780001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:17.780001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:17.780001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:17.780001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:17.780001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:17.780001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:17.780001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:17.780001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:17.780967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:17.780967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:17.780967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:17.780967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:17.780967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:17.780967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:17.780967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:17.780967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:17.780967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:17.780967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:17.780967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:17.780967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:17.780967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:17.780967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:17.780967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:17.780967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:17.781844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:17.781844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:17.781844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:17.781844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:17.781844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:17.781844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:17.781844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:17.781844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:17.781844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:17.781844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:17.781844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:17.781844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:17.781844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 375/736] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:17.781844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c9372-9f51-4f06-ac6a-b1c0b236d349 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.823698 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239de6aa-5588-4c6c-a873-85df3f0b2702 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.861659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e56e3-abbf-444a-84f9-b73b5ab930f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.862295 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e56e3-abbf-444a-84f9-b73b5ab930f0 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.862577 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e56e3-abbf-444a-84f9-b73b5ab930f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.863278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e56e3-abbf-444a-84f9-b73b5ab930f0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.872390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e56e3-abbf-444a-84f9-b73b5ab930f0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:17.873886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e56e3-abbf-444a-84f9-b73b5ab930f0 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:17.890789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-5d5e56e3-abbf-444a-84f9-b73b5ab930f0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:17.890789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:17.890789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:17.890789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:17.890789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:17.890789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:17.890789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:17.890789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:17.890789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:17.890789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:17.890789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:17.890789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:17.890789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:17.890789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:17.890789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:17.890789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:17.891832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:17.892783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:17.892783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:17.892783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:17.892783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:17.892783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:17.892783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:17.892783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:17.892783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:17.892783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:17.892783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:17.893357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c9372-9f51-4f06-ac6a-b1c0b236d349 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:17.894493 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:17.896851 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 362/737] 10.222.0.58 () {56 vars in 1117 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:17.898330 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778e495-b6a5-450b-afee-02c68d00922d None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.898651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778e495-b6a5-450b-afee-02c68d00922d None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.898915 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778e495-b6a5-450b-afee-02c68d00922d None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.900355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 363/738] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:17.904961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778e495-b6a5-450b-afee-02c68d00922d None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:17.934394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a8a12-e909-4560-bf96-9bb4bdb4c746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:17.935784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a8a12-e909-4560-bf96-9bb4bdb4c746 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:17.936492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a8a12-e909-4560-bf96-9bb4bdb4c746 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:17.936954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a8a12-e909-4560-bf96-9bb4bdb4c746 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:17.938415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a8a12-e909-4560-bf96-9bb4bdb4c746 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:17.939025 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a8a12-e909-4560-bf96-9bb4bdb4c746 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:17.940802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a8a12-e909-4560-bf96-9bb4bdb4c746 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:17.949215 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:17.955522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 376/739] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:17.957164 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a8a12-e909-4560-bf96-9bb4bdb4c746 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.020007 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778e495-b6a5-450b-afee-02c68d00922d None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.058958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b959ee-a19c-4c04-9ed7-4a4dfa5b6f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.059769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b959ee-a19c-4c04-9ed7-4a4dfa5b6f05 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.060050 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b959ee-a19c-4c04-9ed7-4a4dfa5b6f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.060325 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b959ee-a19c-4c04-9ed7-4a4dfa5b6f05 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.060986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b959ee-a19c-4c04-9ed7-4a4dfa5b6f05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:18.061289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b959ee-a19c-4c04-9ed7-4a4dfa5b6f05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.062837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b959ee-a19c-4c04-9ed7-4a4dfa5b6f05 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.071662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a8a12-e909-4560-bf96-9bb4bdb4c746 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.077309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 364/740] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:18.077309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e58993-5fde-40d6-b184-a7c6e526c2a7 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.077309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e58993-5fde-40d6-b184-a7c6e526c2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.077309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e58993-5fde-40d6-b184-a7c6e526c2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.093445 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-42e58993-5fde-40d6-b184-a7c6e526c2a7 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:18.094739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-42e58993-5fde-40d6-b184-a7c6e526c2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:18.111269 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743b29f-49e0-4ea8-b539-1bf73dc65a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.111999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743b29f-49e0-4ea8-b539-1bf73dc65a8b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.112350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743b29f-49e0-4ea8-b539-1bf73dc65a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.112631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743b29f-49e0-4ea8-b539-1bf73dc65a8b None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.113364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743b29f-49e0-4ea8-b539-1bf73dc65a8b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:18.113670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743b29f-49e0-4ea8-b539-1bf73dc65a8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.115096 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743b29f-49e0-4ea8-b539-1bf73dc65a8b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.133481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 377/741] 10.222.0.58 () {56 vars in 1149 bytes} [Wed Aug 24 12:20:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:18.134390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743b29f-49e0-4ea8-b539-1bf73dc65a8b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.156836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:18.171864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 365/742] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 12:20:18.185886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e58993-5fde-40d6-b184-a7c6e526c2a7 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.227651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:18.247908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743b29f-49e0-4ea8-b539-1bf73dc65a8b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.284981 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e61d36-bc67-44e4-bc0a-1d280ee0844c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.285549 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e61d36-bc67-44e4-bc0a-1d280ee0844c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.285831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e61d36-bc67-44e4-bc0a-1d280ee0844c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.286194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e61d36-bc67-44e4-bc0a-1d280ee0844c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.294335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e61d36-bc67-44e4-bc0a-1d280ee0844c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.295977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e61d36-bc67-44e4-bc0a-1d280ee0844c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.309264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-65e61d36-bc67-44e4-bc0a-1d280ee0844c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:18.309264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:18.309264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:18.309264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:18.309264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:18.309264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:18.309264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:18.309264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:18.309264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:18.309264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:18.309264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:18.309264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:18.309264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:18.309264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:18.309264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:18.309264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:18.310240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:18.311194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:18.311194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:18.311194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:18.311194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:18.311194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:18.311194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:18.311194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:18.311194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:18.311194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:18.311194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:18.311194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 378/743] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:18.311837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e61d36-bc67-44e4-bc0a-1d280ee0844c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.323887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e58993-5fde-40d6-b184-a7c6e526c2a7 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.372886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe2802-2d6c-4685-b678-08257553e211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.374267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe2802-2d6c-4685-b678-08257553e211 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.374756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe2802-2d6c-4685-b678-08257553e211 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.375233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe2802-2d6c-4685-b678-08257553e211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.390019 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe2802-2d6c-4685-b678-08257553e211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.392638 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe2802-2d6c-4685-b678-08257553e211 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.408373 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:18.417377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-f7fe2802-2d6c-4685-b678-08257553e211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:18.417377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:18.417377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:18.417377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:18.417377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:18.417377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:18.417377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:18.417377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:18.417377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:18.417377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:18.417377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:18.417377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:18.417377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:18.417377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:18.417377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:18.417377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:18.418357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:18.419258 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:18.419258 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:18.419258 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:18.421273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 366/744] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:18.422687 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe2802-2d6c-4685-b678-08257553e211 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.432177 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e61d36-bc67-44e4-bc0a-1d280ee0844c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.468979 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c3f5a-36f5-4286-94a7-f23ee790fdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.469650 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c3f5a-36f5-4286-94a7-f23ee790fdf4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.469936 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c3f5a-36f5-4286-94a7-f23ee790fdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.470207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c3f5a-36f5-4286-94a7-f23ee790fdf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.516718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:18.543265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe2802-2d6c-4685-b678-08257553e211 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.548655 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c3f5a-36f5-4286-94a7-f23ee790fdf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.550137 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c3f5a-36f5-4286-94a7-f23ee790fdf4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.590987 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629fab4-9160-4895-883c-c8263753f9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.591345 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629fab4-9160-4895-883c-c8263753f9b4 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.591648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629fab4-9160-4895-883c-c8263753f9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.591924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629fab4-9160-4895-883c-c8263753f9b4 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.592661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629fab4-9160-4895-883c-c8263753f9b4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:18.592970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629fab4-9160-4895-883c-c8263753f9b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.594398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5629fab4-9160-4895-883c-c8263753f9b4 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.614107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 367/745] 10.222.0.58 () {56 vars in 1135 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:18.614866 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629fab4-9160-4895-883c-c8263753f9b4 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.640371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:18.663305 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 379/746] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 12:20:18.664449 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c3f5a-36f5-4286-94a7-f23ee790fdf4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.727532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629fab4-9160-4895-883c-c8263753f9b4 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.765154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d969ca4-f358-42e0-a6f8-8017bbbe6ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.765758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d969ca4-f358-42e0-a6f8-8017bbbe6ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.766037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d969ca4-f358-42e0-a6f8-8017bbbe6ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.766307 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d969ca4-f358-42e0-a6f8-8017bbbe6ec4 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.766943 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d969ca4-f358-42e0-a6f8-8017bbbe6ec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:18.767236 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d969ca4-f358-42e0-a6f8-8017bbbe6ec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.768639 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d969ca4-f358-42e0-a6f8-8017bbbe6ec4 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.769497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c3f5a-36f5-4286-94a7-f23ee790fdf4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.777361 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:18.785735 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 368/747] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:18.786571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d969ca4-f358-42e0-a6f8-8017bbbe6ec4 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.806995 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe83247-b51b-4b5b-ab62-a95cd33c25b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.807887 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe83247-b51b-4b5b-ab62-a95cd33c25b1 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.808444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe83247-b51b-4b5b-ab62-a95cd33c25b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.808889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe83247-b51b-4b5b-ab62-a95cd33c25b1 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.810107 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe83247-b51b-4b5b-ab62-a95cd33c25b1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:18.812089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe83247-b51b-4b5b-ab62-a95cd33c25b1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:18.831457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 380/748] 10.222.0.58 () {60 vars in 1141 bytes} [Wed Aug 24 12:20:18 2022] POST /identity/v3/services => generated 240 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:18.832936 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe83247-b51b-4b5b-ab62-a95cd33c25b1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:18.911088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:18.919495 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d969ca4-f358-42e0-a6f8-8017bbbe6ec4 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:18.973635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7858428-e05a-4d71-957a-8d8d08a45c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:18.974270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7858428-e05a-4d71-957a-8d8d08a45c53 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:18.974539 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7858428-e05a-4d71-957a-8d8d08a45c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:18.974811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7858428-e05a-4d71-957a-8d8d08a45c53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:18.983001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe83247-b51b-4b5b-ab62-a95cd33c25b1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:19.035502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d190ed8-7496-4e42-b315-8ab8ac949e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:19.036349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d190ed8-7496-4e42-b315-8ab8ac949e06 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.036752 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d190ed8-7496-4e42-b315-8ab8ac949e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.037016 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d190ed8-7496-4e42-b315-8ab8ac949e06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.054775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d190ed8-7496-4e42-b315-8ab8ac949e06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:19.056402 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d190ed8-7496-4e42-b315-8ab8ac949e06 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:19.060843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:19.068200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7858428-e05a-4d71-957a-8d8d08a45c53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:19.069598 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7858428-e05a-4d71-957a-8d8d08a45c53 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:19.077190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-2d190ed8-7496-4e42-b315-8ab8ac949e06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:19.077190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:19.077190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:19.077190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:19.077190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:19.077190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:19.077190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:19.077190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:19.077190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:19.077190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:19.077190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:19.077190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:19.077190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:19.077190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:19.077190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:19.077190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:19.078383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:19.079485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:19.079485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:19.079485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:19.081863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 381/749] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:19.083250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d190ed8-7496-4e42-b315-8ab8ac949e06 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.187789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 369/750] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:20:19.188852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7858428-e05a-4d71-957a-8d8d08a45c53 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.208736 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:19.220894 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d190ed8-7496-4e42-b315-8ab8ac949e06 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:19.270566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643640d5-a75c-4ed5-b0c7-5792fe05c0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:19.272310 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643640d5-a75c-4ed5-b0c7-5792fe05c0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.272772 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643640d5-a75c-4ed5-b0c7-5792fe05c0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.273196 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643640d5-a75c-4ed5-b0c7-5792fe05c0b8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.283206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643640d5-a75c-4ed5-b0c7-5792fe05c0b8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:19.285638 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643640d5-a75c-4ed5-b0c7-5792fe05c0b8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:19.298474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:19.299954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-643640d5-a75c-4ed5-b0c7-5792fe05c0b8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:19.299954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:19.299954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:19.299954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:19.299954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:19.299954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:19.299954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:19.299954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:19.299954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:19.299954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:19.299954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:19.299954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:19.299954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:19.299954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:19.299954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:19.299954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:19.301115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:19.302316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:19.302316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:19.302316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:19.302316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:19.302316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:19.302316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:19.302316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:19.302316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:19.302316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:19.302316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:19.308819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 382/751] 10.222.0.58 () {56 vars in 1093 bytes} [Wed Aug 24 12:20:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:19.308819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643640d5-a75c-4ed5-b0c7-5792fe05c0b8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.316069 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7858428-e05a-4d71-957a-8d8d08a45c53 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:19.373240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989ddb1-2a61-44d9-92cf-6f20b4b30b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:19.374310 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989ddb1-2a61-44d9-92cf-6f20b4b30b26 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.374682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989ddb1-2a61-44d9-92cf-6f20b4b30b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.375483 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989ddb1-2a61-44d9-92cf-6f20b4b30b26 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.376509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989ddb1-2a61-44d9-92cf-6f20b4b30b26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:19.376910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989ddb1-2a61-44d9-92cf-6f20b4b30b26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:19.379254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989ddb1-2a61-44d9-92cf-6f20b4b30b26 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:19.395921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 370/752] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:19.396967 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989ddb1-2a61-44d9-92cf-6f20b4b30b26 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.424432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643640d5-a75c-4ed5-b0c7-5792fe05c0b8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:19.461523 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdf263-5035-490e-b648-1a5b298e9a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:19.462189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdf263-5035-490e-b648-1a5b298e9a43 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.462472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdf263-5035-490e-b648-1a5b298e9a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.462751 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebdf263-5035-490e-b648-1a5b298e9a43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.474709 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebdf263-5035-490e-b648-1a5b298e9a43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:19.476224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebdf263-5035-490e-b648-1a5b298e9a43 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:19.491394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-0ebdf263-5035-490e-b648-1a5b298e9a43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:19.491394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:19.491394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:19.491394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:19.491394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:19.491394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:19.491394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:19.491394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:19.491394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:19.491394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:19.491394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:19.491394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:19.491394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:19.491394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:19.491394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:19.491394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:19.492403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:19.493308 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:19.493308 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:19.493308 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:19.493308 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 383/753] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:19.495668 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdf263-5035-490e-b648-1a5b298e9a43 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.498346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:19.509554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989ddb1-2a61-44d9-92cf-6f20b4b30b26 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:19.547244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5db33-f300-4fb4-9ac7-963d3cec1ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:19.547885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5db33-f300-4fb4-9ac7-963d3cec1ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.548168 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5db33-f300-4fb4-9ac7-963d3cec1ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.548443 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5db33-f300-4fb4-9ac7-963d3cec1ab3 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.549176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5db33-f300-4fb4-9ac7-963d3cec1ab3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:19.549484 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5db33-f300-4fb4-9ac7-963d3cec1ab3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:19.550925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5db33-f300-4fb4-9ac7-963d3cec1ab3 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:19.569637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 371/754] 10.222.0.58 () {56 vars in 1133 bytes} [Wed Aug 24 12:20:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:19.570771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4dd55-b33f-4048-bb6a-7fa3ce359457 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.571060 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4dd55-b33f-4048-bb6a-7fa3ce359457 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.571337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4dd55-b33f-4048-bb6a-7fa3ce359457 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.575658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 372/755] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:19.575658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4dd55-b33f-4048-bb6a-7fa3ce359457 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.593095 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:19.611824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebdf263-5035-490e-b648-1a5b298e9a43 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:19.649766 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6f958-cbe2-4a70-86a4-b1f675349be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:19.650562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6f958-cbe2-4a70-86a4-b1f675349be0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.650817 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6f958-cbe2-4a70-86a4-b1f675349be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.651208 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6f958-cbe2-4a70-86a4-b1f675349be0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.657979 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a6f958-cbe2-4a70-86a4-b1f675349be0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:19.659419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a6f958-cbe2-4a70-86a4-b1f675349be0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:19.671260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e8a6f958-cbe2-4a70-86a4-b1f675349be0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:19.671260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:19.671260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:19.671260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:19.671260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:19.671260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:19.671260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:19.671260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:19.671260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:19.671260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:19.671260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:19.671260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:19.671260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:19.671260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:19.671260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:19.672252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:19.674664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:19.674664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:19.674664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:19.674664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:19.674664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:19.674664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:19.674664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:19.674664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:19.677204 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 384/756] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:19.677909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6f958-cbe2-4a70-86a4-b1f675349be0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.699355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4dd55-b33f-4048-bb6a-7fa3ce359457 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:19.739940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035b3d7-9a85-40e9-9465-de1c7d84de60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:19.740656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035b3d7-9a85-40e9-9465-de1c7d84de60 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.740990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035b3d7-9a85-40e9-9465-de1c7d84de60 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.741355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035b3d7-9a85-40e9-9465-de1c7d84de60 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.742004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035b3d7-9a85-40e9-9465-de1c7d84de60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:19.742308 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035b3d7-9a85-40e9-9465-de1c7d84de60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:19.743809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035b3d7-9a85-40e9-9465-de1c7d84de60 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:19.744889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:19.756961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 373/757] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:19.758297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0b6d5-5a5a-4344-9b34-e4a3627ceab9 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.758650 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0b6d5-5a5a-4344-9b34-e4a3627ceab9 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.758994 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0b6d5-5a5a-4344-9b34-e4a3627ceab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.777185 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-e8d0b6d5-5a5a-4344-9b34-e4a3627ceab9 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:19.778497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e8d0b6d5-5a5a-4344-9b34-e4a3627ceab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:19.787824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6f958-cbe2-4a70-86a4-b1f675349be0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:19.822296 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3120b64-b782-4018-a234-cf77619b3d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:19.822879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3120b64-b782-4018-a234-cf77619b3d28 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:19.823163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3120b64-b782-4018-a234-cf77619b3d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:19.823441 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3120b64-b782-4018-a234-cf77619b3d28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:19.834444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3120b64-b782-4018-a234-cf77619b3d28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:19.835925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3120b64-b782-4018-a234-cf77619b3d28 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:19.849937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e3120b64-b782-4018-a234-cf77619b3d28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:19.849937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:19.849937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:19.849937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:19.849937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:19.849937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:19.849937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:19.849937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:19.849937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:19.849937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:19.849937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:19.849937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:19.849937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:19.849937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:19.849937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:19.849937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:19.850915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:19.851879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:19.851879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:19.851879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:19.851879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 385/758] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:19.851879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3120b64-b782-4018-a234-cf77619b3d28 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.853062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 374/759] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:19.854672 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0b6d5-5a5a-4344-9b34-e4a3627ceab9 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:19.858225 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:19.979408 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0b6d5-5a5a-4344-9b34-e4a3627ceab9 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:19.995493 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3120b64-b782-4018-a234-cf77619b3d28 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.014828 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:20.030027 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdac04b-9bf9-46f9-9071-240628e4376c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.030711 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdac04b-9bf9-46f9-9071-240628e4376c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.030992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdac04b-9bf9-46f9-9071-240628e4376c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.031259 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdac04b-9bf9-46f9-9071-240628e4376c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.039243 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d0073-f48b-436c-bae2-d2aef4ba1de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.040173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d0073-f48b-436c-bae2-d2aef4ba1de3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.040635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdac04b-9bf9-46f9-9071-240628e4376c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.040836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d0073-f48b-436c-bae2-d2aef4ba1de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.041855 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d0073-f48b-436c-bae2-d2aef4ba1de3 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.042679 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdac04b-9bf9-46f9-9071-240628e4376c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.043423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d0073-f48b-436c-bae2-d2aef4ba1de3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:20.043948 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d0073-f48b-436c-bae2-d2aef4ba1de3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.045617 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d0073-f48b-436c-bae2-d2aef4ba1de3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.055205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-0cdac04b-9bf9-46f9-9071-240628e4376c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:20.055205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:20.055205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:20.055205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:20.055205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:20.055205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:20.055205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:20.055205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:20.055205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:20.055205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:20.055205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:20.055205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:20.055205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:20.055205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:20.055205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:20.056176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:20.062301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:20.062301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:20.062301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:20.062301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:20.062301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:20.062301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:20.062301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:20.062301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:20.062301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 375/760] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:20.062301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdac04b-9bf9-46f9-9071-240628e4376c None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.067224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 386/761] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:20.068349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d0073-f48b-436c-bae2-d2aef4ba1de3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.130825 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:20.173388 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdac04b-9bf9-46f9-9071-240628e4376c None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.182465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d0073-f48b-436c-bae2-d2aef4ba1de3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.210736 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc23c7a-4026-4650-a4a0-b1b6aba86519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.211347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc23c7a-4026-4650-a4a0-b1b6aba86519 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.211651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc23c7a-4026-4650-a4a0-b1b6aba86519 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.211928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc23c7a-4026-4650-a4a0-b1b6aba86519 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.212574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc23c7a-4026-4650-a4a0-b1b6aba86519 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:20.213397 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc23c7a-4026-4650-a4a0-b1b6aba86519 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.214773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc23c7a-4026-4650-a4a0-b1b6aba86519 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.226794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 376/762] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:20.227589 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc23c7a-4026-4650-a4a0-b1b6aba86519 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.234286 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e63e6-7dbd-492a-adca-0cc61ec57a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.235687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e63e6-7dbd-492a-adca-0cc61ec57a69 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.237010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e63e6-7dbd-492a-adca-0cc61ec57a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.237517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e63e6-7dbd-492a-adca-0cc61ec57a69 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.238404 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e63e6-7dbd-492a-adca-0cc61ec57a69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:20.238895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e63e6-7dbd-492a-adca-0cc61ec57a69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.240537 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e63e6-7dbd-492a-adca-0cc61ec57a69 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.253752 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 387/763] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:20.259135 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e63e6-7dbd-492a-adca-0cc61ec57a69 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.303445 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:20.338633 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc23c7a-4026-4650-a4a0-b1b6aba86519 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.364432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:20.376409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9279c-cc31-4348-8540-c763aa401afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.377012 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9279c-cc31-4348-8540-c763aa401afe None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.377294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9279c-cc31-4348-8540-c763aa401afe None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.377565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9279c-cc31-4348-8540-c763aa401afe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.378287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9279c-cc31-4348-8540-c763aa401afe None admin] RBAC: Adding query filter params (role.id=5a12926bd458420e81e72ecf825d67ae, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=adc6a2b3f6bd45178cc7ffb181c3c016) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:20.378593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9279c-cc31-4348-8540-c763aa401afe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.378765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e63e6-7dbd-492a-adca-0cc61ec57a69 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.380243 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9279c-cc31-4348-8540-c763aa401afe None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.386480 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 377/764] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/role_assignments?user.id=adc6a2b3f6bd45178cc7ffb181c3c016&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=5a12926bd458420e81e72ecf825d67ae => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:20.387030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9279c-cc31-4348-8540-c763aa401afe None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.439424 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d398b18-3e3b-474b-bf8d-c0d037eda089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.440524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d398b18-3e3b-474b-bf8d-c0d037eda089 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.441311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d398b18-3e3b-474b-bf8d-c0d037eda089 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.441801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d398b18-3e3b-474b-bf8d-c0d037eda089 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.443224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d398b18-3e3b-474b-bf8d-c0d037eda089 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.445423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d398b18-3e3b-474b-bf8d-c0d037eda089 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-5d398b18-3e3b-474b-bf8d-c0d037eda089 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:20.459816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:20.460898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:20.460898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:20.460898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:20.460898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:20.460898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:20.460898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:20.460898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:20.460898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:20.460898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:20.460898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:20.460898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:20.460898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:20.460898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:20.460898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:20.460898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:20.460898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:20.461732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:20.461732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:20.461732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:20.461732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:20.461732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:20.461732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:20.461732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:20.461732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:20.461732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:20.461732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:20.461732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:20.461732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:20.461732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:20.461732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:20.470237 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 388/765] 10.222.0.58 () {56 vars in 1129 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:20.470237 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d398b18-3e3b-474b-bf8d-c0d037eda089 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.511807 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9279c-cc31-4348-8540-c763aa401afe None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.552806 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefebc2b-bc4b-4550-a567-22515a5e4dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.553476 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefebc2b-bc4b-4550-a567-22515a5e4dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.553755 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefebc2b-bc4b-4550-a567-22515a5e4dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.554091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefebc2b-bc4b-4550-a567-22515a5e4dd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.561621 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefebc2b-bc4b-4550-a567-22515a5e4dd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.563122 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefebc2b-bc4b-4550-a567-22515a5e4dd6 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.574597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-cefebc2b-bc4b-4550-a567-22515a5e4dd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:20.574597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:20.574597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:20.574597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:20.574597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:20.574597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:20.574597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:20.574597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:20.574597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:20.574597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:20.574597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:20.574597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:20.574597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:20.574597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:20.574597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:20.576145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:20.577360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:20.577360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:20.577360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:20.577360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:20.577360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:20.577360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:20.577360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:20.577360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:20.580766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 378/766] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:20.581595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefebc2b-bc4b-4550-a567-22515a5e4dd6 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.609083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:20.614491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d398b18-3e3b-474b-bf8d-c0d037eda089 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.636783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:20.661653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d5110-e015-4c6d-9196-25d7da04effc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.662608 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d5110-e015-4c6d-9196-25d7da04effc None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.663102 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d5110-e015-4c6d-9196-25d7da04effc None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.663582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d5110-e015-4c6d-9196-25d7da04effc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.664520 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d5110-e015-4c6d-9196-25d7da04effc None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=67756e31aeee48a897cfa8bd35a10119) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:20.665021 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d5110-e015-4c6d-9196-25d7da04effc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.668208 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d5110-e015-4c6d-9196-25d7da04effc None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.675407 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 389/767] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/role_assignments?user.id=67756e31aeee48a897cfa8bd35a10119&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:20.677427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d5110-e015-4c6d-9196-25d7da04effc None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.701959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefebc2b-bc4b-4550-a567-22515a5e4dd6 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.761622 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4392b-2f08-475c-9651-d707b62b110c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.763377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4392b-2f08-475c-9651-d707b62b110c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.766990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4392b-2f08-475c-9651-d707b62b110c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.767587 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4392b-2f08-475c-9651-d707b62b110c None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.768503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4392b-2f08-475c-9651-d707b62b110c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:20.769068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4392b-2f08-475c-9651-d707b62b110c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.770680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4392b-2f08-475c-9651-d707b62b110c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.791972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 379/768] 10.222.0.58 () {56 vars in 1124 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:20.799650 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4392b-2f08-475c-9651-d707b62b110c None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.866279 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d5110-e015-4c6d-9196-25d7da04effc None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.908729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828813b-7a3f-4f60-825c-29bfce42f4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:20.909256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828813b-7a3f-4f60-825c-29bfce42f4bf None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.909539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828813b-7a3f-4f60-825c-29bfce42f4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.909814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828813b-7a3f-4f60-825c-29bfce42f4bf None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.910507 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828813b-7a3f-4f60-825c-29bfce42f4bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:20.910811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828813b-7a3f-4f60-825c-29bfce42f4bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:20.912279 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828813b-7a3f-4f60-825c-29bfce42f4bf None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:20.925560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 390/769] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:20.926872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95627561-0477-45ee-8e83-fee3d8f8d387 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:20.927156 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95627561-0477-45ee-8e83-fee3d8f8d387 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:20.927425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95627561-0477-45ee-8e83-fee3d8f8d387 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:20.928806 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 391/770] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:20.933023 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95627561-0477-45ee-8e83-fee3d8f8d387 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:20.959582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:20.966711 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4392b-2f08-475c-9651-d707b62b110c None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:20.984978 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:21.015074 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a6e63-490c-4cd8-8223-69b764d87862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:21.016241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a6e63-490c-4cd8-8223-69b764d87862 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:21.016522 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a6e63-490c-4cd8-8223-69b764d87862 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:21.016793 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a6e63-490c-4cd8-8223-69b764d87862 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:21.017425 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a6e63-490c-4cd8-8223-69b764d87862 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:21.019564 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a6e63-490c-4cd8-8223-69b764d87862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:21.020927 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a6e63-490c-4cd8-8223-69b764d87862 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:21.031777 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 380/771] 10.222.0.58 () {56 vars in 1124 bytes} [Wed Aug 24 12:20:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:21.033051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6cfba-84ee-4d57-a6e8-3e7b0e6818c0 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:21.033335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6cfba-84ee-4d57-a6e8-3e7b0e6818c0 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:21.033606 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6cfba-84ee-4d57-a6e8-3e7b0e6818c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:21.049828 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95627561-0477-45ee-8e83-fee3d8f8d387 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:21.052362 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-d8e6cfba-84ee-4d57-a6e8-3e7b0e6818c0 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:21.053651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d8e6cfba-84ee-4d57-a6e8-3e7b0e6818c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:21.090161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b089e99-565e-477a-82fe-c8fde949c236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:21.091344 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b089e99-565e-477a-82fe-c8fde949c236 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:21.091869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b089e99-565e-477a-82fe-c8fde949c236 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:21.092384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b089e99-565e-477a-82fe-c8fde949c236 None admin] PATH_INFO: `/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/311dbed3ab0b44a7830a93227ff9ad33/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:21.094680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b089e99-565e-477a-82fe-c8fde949c236 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user_id=311dbed3ab0b44a7830a93227ff9ad33, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:21.096502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b089e99-565e-477a-82fe-c8fde949c236 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:21.114151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 392/772] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:20 2022] PUT /identity/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/311dbed3ab0b44a7830a93227ff9ad33/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:20:21.115380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b089e99-565e-477a-82fe-c8fde949c236 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:21.131829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 381/773] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:21.219224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:21.239052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b089e99-565e-477a-82fe-c8fde949c236 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:21.288414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f442ea5-93e1-40af-9093-65a656f5beb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:21.289037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f442ea5-93e1-40af-9093-65a656f5beb6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:21.289325 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f442ea5-93e1-40af-9093-65a656f5beb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:21.289596 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f442ea5-93e1-40af-9093-65a656f5beb6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:21.290304 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f442ea5-93e1-40af-9093-65a656f5beb6 None admin] RBAC: Adding query filter params (interface=public, service_id=07cf75c809f64154aa9ca3996ba0eeb7, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:21.290608 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f442ea5-93e1-40af-9093-65a656f5beb6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:21.292831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f442ea5-93e1-40af-9093-65a656f5beb6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:21.305048 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 393/774] 10.222.0.58 () {56 vars in 1251 bytes} [Wed Aug 24 12:20:21 2022] GET /identity/v3/endpoints?service_id=07cf75c809f64154aa9ca3996ba0eeb7&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:21.311387 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6cfba-84ee-4d57-a6e8-3e7b0e6818c0 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:21.439454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6cfba-84ee-4d57-a6e8-3e7b0e6818c0 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:21.447151 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:21.468743 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f442ea5-93e1-40af-9093-65a656f5beb6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:21.489802 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48e965-2182-4875-aced-1cdff9042a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:21.490350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48e965-2182-4875-aced-1cdff9042a7d None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:21.490631 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48e965-2182-4875-aced-1cdff9042a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:21.490904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48e965-2182-4875-aced-1cdff9042a7d None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:21.491518 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48e965-2182-4875-aced-1cdff9042a7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:21.492960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48e965-2182-4875-aced-1cdff9042a7d None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:21.499343 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 382/775] 10.222.0.58 () {56 vars in 1087 bytes} [Wed Aug 24 12:20:21 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:21.625623 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f442ea5-93e1-40af-9093-65a656f5beb6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:21.662093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3300b6-011d-460b-8002-e68953558137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:21.662730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3300b6-011d-460b-8002-e68953558137 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:21.663013 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3300b6-011d-460b-8002-e68953558137 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:21.663291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3300b6-011d-460b-8002-e68953558137 None admin] PATH_INFO: `/v3/services/network` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:21.663914 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3300b6-011d-460b-8002-e68953558137 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:21.665285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3300b6-011d-460b-8002-e68953558137 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6a3300b6-011d-460b-8002-e68953558137 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:21.674852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:21.675895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:21.675895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:21.675895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:21.675895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:21.675895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:21.675895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:21.675895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:21.675895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:21.675895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:21.675895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:21.675895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:21.675895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:21.675895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:21.675895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:21.675895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:21.675895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:21.676952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:21.676952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:21.676952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:21.676952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:21.676952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:21.676952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:21.676952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:21.676952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:21.676952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:21.676952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:21.676952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:21.676952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:21.676952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:21.676952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:21.676952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 394/776] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:21 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:21.684781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48e965-2182-4875-aced-1cdff9042a7d None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:21.803670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48e965-2182-4875-aced-1cdff9042a7d None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:21.843745 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde909b-dc42-46bc-8323-dd9a231af570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:21.844360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde909b-dc42-46bc-8323-dd9a231af570 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:21.844642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde909b-dc42-46bc-8323-dd9a231af570 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:21.844914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde909b-dc42-46bc-8323-dd9a231af570 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:21.845550 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde909b-dc42-46bc-8323-dd9a231af570 None admin] RBAC: Adding query filter params (name=network) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:21.845852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde909b-dc42-46bc-8323-dd9a231af570 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:21.847234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde909b-dc42-46bc-8323-dd9a231af570 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:21.852482 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 383/777] 10.222.0.58 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:21.860747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3300b6-011d-460b-8002-e68953558137 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:21.904037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:21.987845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3300b6-011d-460b-8002-e68953558137 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:22.035907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cdfd1-b06c-4840-af77-d1cca19db85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:22.036521 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cdfd1-b06c-4840-af77-d1cca19db85d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:22.036803 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cdfd1-b06c-4840-af77-d1cca19db85d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:22.037129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65cdfd1-b06c-4840-af77-d1cca19db85d None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:22.037773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65cdfd1-b06c-4840-af77-d1cca19db85d None admin] RBAC: Adding query filter params (type=network) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:22.038078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65cdfd1-b06c-4840-af77-d1cca19db85d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:22.039446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65cdfd1-b06c-4840-af77-d1cca19db85d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:22.047859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 395/778] 10.222.0.58 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:22.053897 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde909b-dc42-46bc-8323-dd9a231af570 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:22.095928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:22.177003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde909b-dc42-46bc-8323-dd9a231af570 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:22.226221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14481e78-1c54-428b-ad1f-f5542d563c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:22.226838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14481e78-1c54-428b-ad1f-f5542d563c68 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:22.227119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14481e78-1c54-428b-ad1f-f5542d563c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:22.227393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14481e78-1c54-428b-ad1f-f5542d563c68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:22.228137 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14481e78-1c54-428b-ad1f-f5542d563c68 None admin] RBAC: Adding query filter params (interface=public, service_id=e44f2d23b5f0460fb7f4d484ab8b8885, region_id=RegionOne) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:22.228527 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14481e78-1c54-428b-ad1f-f5542d563c68 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:22.230720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14481e78-1c54-428b-ad1f-f5542d563c68 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:22.241971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 384/779] 10.222.0.58 () {56 vars in 1251 bytes} [Wed Aug 24 12:20:22 2022] GET /identity/v3/endpoints?service_id=e44f2d23b5f0460fb7f4d484ab8b8885&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:22.248073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cdfd1-b06c-4840-af77-d1cca19db85d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:22.349618 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:22.379111 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65cdfd1-b06c-4840-af77-d1cca19db85d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:22.433623 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ea9d3-460b-4adb-a094-de737a56d239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:22.434345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ea9d3-460b-4adb-a094-de737a56d239 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:22.434628 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ea9d3-460b-4adb-a094-de737a56d239 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:22.434908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ea9d3-460b-4adb-a094-de737a56d239 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:22.435603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ea9d3-460b-4adb-a094-de737a56d239 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:22.437004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888ea9d3-460b-4adb-a094-de737a56d239 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:22.448534 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 396/780] 10.222.0.58 () {56 vars in 1087 bytes} [Wed Aug 24 12:20:22 2022] GET /identity/v3/services => generated 772 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:22.738751 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a8611-7953-4797-9724-6d0c685296c5 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:22.739854 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a8611-7953-4797-9724-6d0c685296c5 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:22.740345 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a8611-7953-4797-9724-6d0c685296c5 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:22.742204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 385/781] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:22.748391 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf236e-12f3-4f22-989b-88072fed60d4 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:22.748684 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf236e-12f3-4f22-989b-88072fed60d4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:22.748966 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf236e-12f3-4f22-989b-88072fed60d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:22.770641 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-07cf236e-12f3-4f22-989b-88072fed60d4 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:22.772209 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-07cf236e-12f3-4f22-989b-88072fed60d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:22.857845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 397/782] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:22.974707 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8c388-7cdb-4ad4-9f50-add7dbfbf16c None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:22.975344 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8c388-7cdb-4ad4-9f50-add7dbfbf16c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:22.975863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca8c388-7cdb-4ad4-9f50-add7dbfbf16c None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:22.977671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 386/783] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:22.984869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a5ae9-2b15-46d8-96d5-0e40082dc684 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:22.985199 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a5ae9-2b15-46d8-96d5-0e40082dc684 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:22.985425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a5ae9-2b15-46d8-96d5-0e40082dc684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:23.004143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c84a5ae9-2b15-46d8-96d5-0e40082dc684 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:23.005263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c84a5ae9-2b15-46d8-96d5-0e40082dc684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:23.070433 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:23.075529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 398/784] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:23.322854 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8c388-7cdb-4ad4-9f50-add7dbfbf16c None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:23.493282 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a5ae9-2b15-46d8-96d5-0e40082dc684 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:23.560490 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:23.568630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca8c388-7cdb-4ad4-9f50-add7dbfbf16c None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:23.645339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a5ae9-2b15-46d8-96d5-0e40082dc684 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:23.663103 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04ab2a-16ed-40c7-96ab-fa2a3283699d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:23.663828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04ab2a-16ed-40c7-96ab-fa2a3283699d None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:23.664111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04ab2a-16ed-40c7-96ab-fa2a3283699d None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:23.664386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd04ab2a-16ed-40c7-96ab-fa2a3283699d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:23.684067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d653fe5-2f0a-4140-8913-dab040c85aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:23.684769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d653fe5-2f0a-4140-8913-dab040c85aef None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:23.685058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d653fe5-2f0a-4140-8913-dab040c85aef None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:23.685388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d653fe5-2f0a-4140-8913-dab040c85aef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:23.688641 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd04ab2a-16ed-40c7-96ab-fa2a3283699d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:23.698240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-fd04ab2a-16ed-40c7-96ab-fa2a3283699d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:23.698240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:23.698240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:23.698240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:23.698240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:23.698240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:23.698240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:23.698240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:23.698240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:23.698240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:23.698240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:23.698240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:23.698240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:23.698240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:23.698240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:23.698240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:23.699221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:23.699221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:23.699221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:23.699221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:23.699221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:23.699221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:23.699221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:23.699221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:23.699221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:23.699221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:23.699221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:23.699221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:23.699221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:23.699221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:23.699221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:23.699221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:23.700399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:23.700399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:23.700399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:23.700399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:23.700399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:23.700399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:23.700399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:23.700399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:23.700399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:23.700399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:23.700399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:23.700399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:23.705477 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 387/785] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:23.706668 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d653fe5-2f0a-4140-8913-dab040c85aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:23.711034 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04ab2a-16ed-40c7-96ab-fa2a3283699d None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:23.711702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6d653fe5-2f0a-4140-8913-dab040c85aef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:23.711702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:23.711702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:23.711702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:23.711702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:23.711702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:23.711702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:23.711702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:23.711702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:23.711702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:23.711702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:23.711702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:23.711702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:23.711702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:23.711702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:23.711702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:23.714165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:23.714165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:23.714165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:23.714165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:23.714165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:23.714165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:23.714165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:23.714165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:23.714165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:23.714165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:23.714165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:23.714165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:23.714165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:23.714165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:23.714165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:23.714165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:23.716193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:23.716193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:23.716193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:23.716193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:23.716193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:23.716193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:23.716193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:23.716193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:23.716193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:23.716193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:23.716193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:23.716193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:23.716193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 399/786] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:23.719704 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d653fe5-2f0a-4140-8913-dab040c85aef None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:23.787544 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:23.840404 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d653fe5-2f0a-4140-8913-dab040c85aef None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:23.876856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32a2f3-ff6c-4c82-b805-58f9ae60a06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:23.876856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32a2f3-ff6c-4c82-b805-58f9ae60a06e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:23.877159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32a2f3-ff6c-4c82-b805-58f9ae60a06e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:23.877413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32a2f3-ff6c-4c82-b805-58f9ae60a06e None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:23.877931 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32a2f3-ff6c-4c82-b805-58f9ae60a06e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:23.880752 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32a2f3-ff6c-4c82-b805-58f9ae60a06e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:23.880752 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32a2f3-ff6c-4c82-b805-58f9ae60a06e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:23.887938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd04ab2a-16ed-40c7-96ab-fa2a3283699d None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:23.888912 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 400/787] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:23.890161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e155fa1-64f4-403e-a9ef-8d7a9763b481 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:23.890450 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e155fa1-64f4-403e-a9ef-8d7a9763b481 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:23.890728 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e155fa1-64f4-403e-a9ef-8d7a9763b481 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:23.892784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 401/788] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:23.894070 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e155fa1-64f4-403e-a9ef-8d7a9763b481 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:23.929164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f446147-0c50-4f1d-b482-ada51377d40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:23.930026 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f446147-0c50-4f1d-b482-ada51377d40b None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:23.930453 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f446147-0c50-4f1d-b482-ada51377d40b None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:23.930879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f446147-0c50-4f1d-b482-ada51377d40b None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:23.931855 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f446147-0c50-4f1d-b482-ada51377d40b None admin] RBAC: Adding query filter params (name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:23.932363 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f446147-0c50-4f1d-b482-ada51377d40b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:23.934220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f446147-0c50-4f1d-b482-ada51377d40b None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:23.944107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 388/789] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:23.945702 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc9bcb-1d8e-489e-81fc-ab4af2ee983f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:23.946132 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc9bcb-1d8e-489e-81fc-ab4af2ee983f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:23.946549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc9bcb-1d8e-489e-81fc-ab4af2ee983f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:23.967374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-a8dc9bcb-1d8e-489e-81fc-ab4af2ee983f None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:23.969029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a8dc9bcb-1d8e-489e-81fc-ab4af2ee983f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:23.985748 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:24.007379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e155fa1-64f4-403e-a9ef-8d7a9763b481 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:24.030328 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:24.040784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 389/790] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:24.042201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc9bcb-1d8e-489e-81fc-ab4af2ee983f None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:24.050592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4f75a-539a-4a68-bae4-cacdb90132d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:24.051431 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4f75a-539a-4a68-bae4-cacdb90132d5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:24.052252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4f75a-539a-4a68-bae4-cacdb90132d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:24.052679 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4f75a-539a-4a68-bae4-cacdb90132d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:24.067032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f4f75a-539a-4a68-bae4-cacdb90132d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:24.069437 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f4f75a-539a-4a68-bae4-cacdb90132d5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:24.089266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a1f4f75a-539a-4a68-bae4-cacdb90132d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:24.089266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:24.089266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:24.089266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:24.089266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:24.089266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:24.089266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:24.089266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:24.089266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:24.089266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:24.089266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:24.089266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:24.089266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:24.089266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:24.089266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:24.089266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:24.091292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:24.092386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:24.092386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:24.092386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:24.095750 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 402/791] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:24.096879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071ff3f-7700-4f21-ad9d-f8ad83c1c94c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:24.097351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071ff3f-7700-4f21-ad9d-f8ad83c1c94c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:24.097631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071ff3f-7700-4f21-ad9d-f8ad83c1c94c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:24.104057 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 403/792] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:24.105254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071ff3f-7700-4f21-ad9d-f8ad83c1c94c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:24.157460 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc9bcb-1d8e-489e-81fc-ab4af2ee983f None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:24.199149 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba346d-3cc8-4214-9c09-3445ba67a7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:24.199783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba346d-3cc8-4214-9c09-3445ba67a7ff None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:24.200327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba346d-3cc8-4214-9c09-3445ba67a7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:24.200599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba346d-3cc8-4214-9c09-3445ba67a7ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:24.211690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba346d-3cc8-4214-9c09-3445ba67a7ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:24.213464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba346d-3cc8-4214-9c09-3445ba67a7ff None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:24.229903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-25ba346d-3cc8-4214-9c09-3445ba67a7ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:24.229903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:24.229903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:24.229903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:24.229903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:24.229903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:24.229903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:24.229903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:24.229903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:24.229903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:24.229903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:24.229903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:24.229903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:24.229903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:24.229903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:24.229903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:24.231108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:24.232183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:24.232183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:24.232183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:24.232183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 390/793] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:24.233749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24871a-b5ca-4fd5-b904-e941a0a0d2cc None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:24.234118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24871a-b5ca-4fd5-b904-e941a0a0d2cc None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:24.234368 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24871a-b5ca-4fd5-b904-e941a0a0d2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:24.235998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071ff3f-7700-4f21-ad9d-f8ad83c1c94c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:24.241221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:24.255493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-fc24871a-b5ca-4fd5-b904-e941a0a0d2cc None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:24.256749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fc24871a-b5ca-4fd5-b904-e941a0a0d2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:24.280538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51c1d7-e1fb-4471-93d7-4abad229253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:24.281701 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51c1d7-e1fb-4471-93d7-4abad229253a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:24.282569 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51c1d7-e1fb-4471-93d7-4abad229253a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:24.283284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51c1d7-e1fb-4471-93d7-4abad229253a None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:24.284354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51c1d7-e1fb-4471-93d7-4abad229253a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:24.284825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51c1d7-e1fb-4471-93d7-4abad229253a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:24.286999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51c1d7-e1fb-4471-93d7-4abad229253a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:24.303419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 404/794] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:24.304625 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51c1d7-e1fb-4471-93d7-4abad229253a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:24.334534 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 391/795] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:24.340593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24871a-b5ca-4fd5-b904-e941a0a0d2cc None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:24.346062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:24.448463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51c1d7-e1fb-4471-93d7-4abad229253a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:24.490761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47edf9-7fe3-459f-bdb3-7de5a0bfe10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:24.491061 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24871a-b5ca-4fd5-b904-e941a0a0d2cc None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:24.491549 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47edf9-7fe3-459f-bdb3-7de5a0bfe10c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:24.491956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47edf9-7fe3-459f-bdb3-7de5a0bfe10c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:24.492349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47edf9-7fe3-459f-bdb3-7de5a0bfe10c None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:24.493158 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47edf9-7fe3-459f-bdb3-7de5a0bfe10c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:24.493593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47edf9-7fe3-459f-bdb3-7de5a0bfe10c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:24.495097 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47edf9-7fe3-459f-bdb3-7de5a0bfe10c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:24.508383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 405/796] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:24.509346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47edf9-7fe3-459f-bdb3-7de5a0bfe10c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:24.521843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:24.529055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b98885-9a0e-4483-bae0-e538fa469e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:24.530063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b98885-9a0e-4483-bae0-e538fa469e7e None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:24.530350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b98885-9a0e-4483-bae0-e538fa469e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:24.530623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b98885-9a0e-4483-bae0-e538fa469e7e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:24.539162 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b98885-9a0e-4483-bae0-e538fa469e7e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:24.541827 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b98885-9a0e-4483-bae0-e538fa469e7e None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:24.556549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-f1b98885-9a0e-4483-bae0-e538fa469e7e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:24.556549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:24.556549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:24.556549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:24.556549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:24.556549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:24.556549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:24.556549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:24.556549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:24.556549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:24.556549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:24.556549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:24.556549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:24.556549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:24.556549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:24.556549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:24.557852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:24.558853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:24.558853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:24.558853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:24.558853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:24.558853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:24.558853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:24.558853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:24.558853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:24.558853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:24.558853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:24.562871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 392/797] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:24.563756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b98885-9a0e-4483-bae0-e538fa469e7e None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:24.637561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:24.645665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47edf9-7fe3-459f-bdb3-7de5a0bfe10c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:24.673853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b98885-9a0e-4483-bae0-e538fa469e7e None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:24.681583 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b49a1-4542-4a10-9086-7a2ee40ad1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:24.682209 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b49a1-4542-4a10-9086-7a2ee40ad1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:24.682483 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b49a1-4542-4a10-9086-7a2ee40ad1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:24.682757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b49a1-4542-4a10-9086-7a2ee40ad1d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:24.693285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b49a1-4542-4a10-9086-7a2ee40ad1d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:24.699133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-7d6b49a1-4542-4a10-9086-7a2ee40ad1d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:24.699133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:24.699133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:24.699133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:24.699133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:24.699133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:24.699133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:24.699133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:24.699133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:24.699133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:24.699133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:24.699133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:24.699133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:24.699133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:24.699133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:24.699133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:24.700374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:24.700374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:24.700374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:24.700374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:24.700374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:24.700374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:24.700374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:24.700374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:24.700374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:24.700374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:24.700374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:24.700374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:24.700374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:24.700374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:24.700374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:24.700374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:24.701263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:24.701263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:24.701263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:24.701263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:24.701263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:24.701263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:24.701263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:24.701263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:24.701263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:24.701263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:24.701263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:24.701263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:24.701263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 406/798] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:24.701965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b49a1-4542-4a10-9086-7a2ee40ad1d6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:24.716757 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a1ea3-0511-4142-96cd-b5be6a6884ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:24.717384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a1ea3-0511-4142-96cd-b5be6a6884ff None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:24.717821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a1ea3-0511-4142-96cd-b5be6a6884ff None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:24.717979 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a1ea3-0511-4142-96cd-b5be6a6884ff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:24.725642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a1ea3-0511-4142-96cd-b5be6a6884ff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:24.727074 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a1ea3-0511-4142-96cd-b5be6a6884ff None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:24.739023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-048a1ea3-0511-4142-96cd-b5be6a6884ff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:24.739023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:24.739023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:24.739023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:24.739023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:24.739023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:24.739023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:24.739023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:24.739023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:24.739023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:24.739023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:24.739023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:24.739023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:24.739023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:24.739023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:24.739023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:24.740256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:24.741267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:24.741267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:24.741267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:24.741267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:24.741267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:24.741267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:24.741267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:24.741267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:24.741267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:24.741267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:24.741267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 393/799] 10.222.0.58 () {56 vars in 1117 bytes} [Wed Aug 24 12:20:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:24.741267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a1ea3-0511-4142-96cd-b5be6a6884ff None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:24.799200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:24.815982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b49a1-4542-4a10-9086-7a2ee40ad1d6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:24.849821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a1ea3-0511-4142-96cd-b5be6a6884ff None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:24.851525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef710b-7aba-4d66-bda9-26b35fbb02e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:24.852165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef710b-7aba-4d66-bda9-26b35fbb02e4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:24.852445 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef710b-7aba-4d66-bda9-26b35fbb02e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:24.852720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef710b-7aba-4d66-bda9-26b35fbb02e4 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:24.853561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef710b-7aba-4d66-bda9-26b35fbb02e4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:24.853979 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef710b-7aba-4d66-bda9-26b35fbb02e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:24.855419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef710b-7aba-4d66-bda9-26b35fbb02e4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:24.872135 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 407/800] 10.222.0.58 () {56 vars in 1135 bytes} [Wed Aug 24 12:20:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:24.872926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef710b-7aba-4d66-bda9-26b35fbb02e4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:24.888842 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a06049-3aa7-4092-aa99-a8de7f32cfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:24.889484 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a06049-3aa7-4092-aa99-a8de7f32cfbc None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:24.889764 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a06049-3aa7-4092-aa99-a8de7f32cfbc None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:24.890038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a06049-3aa7-4092-aa99-a8de7f32cfbc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:24.890622 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a06049-3aa7-4092-aa99-a8de7f32cfbc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:24.892018 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a06049-3aa7-4092-aa99-a8de7f32cfbc None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:24.892153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-28a06049-3aa7-4092-aa99-a8de7f32cfbc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:24.902065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:24.903282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:24.903282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:24.903282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:24.903282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:24.903282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:24.903282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:24.903282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:24.903282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:24.903282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:24.903282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:24.903282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:24.903282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:24.903282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:24.903282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:24.903282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:24.903282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:24.904374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:24.904374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:24.904374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:24.904374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:24.904374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:24.904374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:24.904374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:24.904374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:24.904374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:24.904374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:24.904374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:24.904374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:24.904374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 12:20:24.904374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:24.904374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 394/801] 10.222.0.58 () {56 vars in 1129 bytes} [Wed Aug 24 12:20:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:24.905382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a06049-3aa7-4092-aa99-a8de7f32cfbc None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:24.981018 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef710b-7aba-4d66-bda9-26b35fbb02e4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.001998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a06049-3aa7-4092-aa99-a8de7f32cfbc None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.015664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb09c9-4f5d-420d-b5e0-5aab6fc1c8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:25.016265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb09c9-4f5d-420d-b5e0-5aab6fc1c8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.016551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb09c9-4f5d-420d-b5e0-5aab6fc1c8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.016827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb09c9-4f5d-420d-b5e0-5aab6fc1c8b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.017484 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb09c9-4f5d-420d-b5e0-5aab6fc1c8b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:25.017784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb09c9-4f5d-420d-b5e0-5aab6fc1c8b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:25.018068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:25.019265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb09c9-4f5d-420d-b5e0-5aab6fc1c8b0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:25.028343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 408/802] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:25.029355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66f6a6-5c20-44bf-af29-34560bb3d3a8 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.029643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66f6a6-5c20-44bf-af29-34560bb3d3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.029921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66f6a6-5c20-44bf-af29-34560bb3d3a8 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.035117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 409/803] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:25.035117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66f6a6-5c20-44bf-af29-34560bb3d3a8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.037800 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a87720-633c-4b05-bdc0-b35c7d4be953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:25.038402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a87720-633c-4b05-bdc0-b35c7d4be953 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.038682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a87720-633c-4b05-bdc0-b35c7d4be953 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.038952 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a87720-633c-4b05-bdc0-b35c7d4be953 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.039715 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a87720-633c-4b05-bdc0-b35c7d4be953 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:25.040020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a87720-633c-4b05-bdc0-b35c7d4be953 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:25.041488 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a87720-633c-4b05-bdc0-b35c7d4be953 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:25.058717 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 395/804] 10.222.0.58 () {56 vars in 1149 bytes} [Wed Aug 24 12:20:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:25.059735 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a87720-633c-4b05-bdc0-b35c7d4be953 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.108329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:25.135536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66f6a6-5c20-44bf-af29-34560bb3d3a8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.159034 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a87720-633c-4b05-bdc0-b35c7d4be953 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.168179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710168c-ab1f-4f7d-8c83-1128da723f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:25.168810 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710168c-ab1f-4f7d-8c83-1128da723f5d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.169094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710168c-ab1f-4f7d-8c83-1128da723f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.169371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710168c-ab1f-4f7d-8c83-1128da723f5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.180305 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710168c-ab1f-4f7d-8c83-1128da723f5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:25.182397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710168c-ab1f-4f7d-8c83-1128da723f5d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:25.193658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8053c7-df71-4a92-a25e-2422fa415bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:25.194253 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8053c7-df71-4a92-a25e-2422fa415bfd None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.194529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8053c7-df71-4a92-a25e-2422fa415bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.194796 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8053c7-df71-4a92-a25e-2422fa415bfd None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.195411 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8053c7-df71-4a92-a25e-2422fa415bfd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:25.195741 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8053c7-df71-4a92-a25e-2422fa415bfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:25.196249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-7710168c-ab1f-4f7d-8c83-1128da723f5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:25.196249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:25.196249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:25.196249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:25.196249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:25.196249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:25.196249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:25.196249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:25.196249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:25.196249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:25.196249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:25.196249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:25.196249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:25.196249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:25.196249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:25.196249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:25.197253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:25.198775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:25.198775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:25.198775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:25.198958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8053c7-df71-4a92-a25e-2422fa415bfd None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:25.199037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 410/805] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:25.199037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710168c-ab1f-4f7d-8c83-1128da723f5d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.203230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 396/806] 10.222.0.58 () {56 vars in 1124 bytes} [Wed Aug 24 12:20:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:25.204411 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a19b8-f05c-4630-9089-88615798ae78 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.204674 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a19b8-f05c-4630-9089-88615798ae78 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.204943 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a19b8-f05c-4630-9089-88615798ae78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.221904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-915a19b8-f05c-4630-9089-88615798ae78 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:25.223083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-915a19b8-f05c-4630-9089-88615798ae78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:25.252977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:25.288921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 397/807] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:25.289738 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a19b8-f05c-4630-9089-88615798ae78 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.303789 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710168c-ab1f-4f7d-8c83-1128da723f5d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.330762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:25.340854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f543239-08d1-47ca-b2e0-a75d5a3df53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:25.341492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f543239-08d1-47ca-b2e0-a75d5a3df53d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.341791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f543239-08d1-47ca-b2e0-a75d5a3df53d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.342061 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f543239-08d1-47ca-b2e0-a75d5a3df53d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.353460 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f543239-08d1-47ca-b2e0-a75d5a3df53d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:25.354921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f543239-08d1-47ca-b2e0-a75d5a3df53d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:25.369959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6f543239-08d1-47ca-b2e0-a75d5a3df53d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:25.369959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:25.369959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:25.369959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:25.369959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:25.369959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:25.369959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:25.369959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:25.369959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:25.369959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:25.369959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:25.369959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:25.369959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:25.369959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:25.369959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:25.369959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:25.370913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:25.371854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:25.371854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:25.371854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:25.371854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 411/808] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:25.372082 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f543239-08d1-47ca-b2e0-a75d5a3df53d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.401112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a19b8-f05c-4630-9089-88615798ae78 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.435515 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a971a1f-ffee-4755-b8ec-88f70859db22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:25.436489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a971a1f-ffee-4755-b8ec-88f70859db22 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.436911 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a971a1f-ffee-4755-b8ec-88f70859db22 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.437183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a971a1f-ffee-4755-b8ec-88f70859db22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.447147 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a971a1f-ffee-4755-b8ec-88f70859db22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:25.449345 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a971a1f-ffee-4755-b8ec-88f70859db22 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:25.464504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-3a971a1f-ffee-4755-b8ec-88f70859db22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:25.464504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:25.464504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:25.464504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:25.464504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:25.464504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:25.464504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:25.464504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:25.464504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:25.464504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:25.464504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:25.464504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:25.464504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:25.464504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:25.464504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:25.464504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:25.465466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:25.466365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:25.466365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:25.466365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:25.466365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 398/809] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:25.466365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a971a1f-ffee-4755-b8ec-88f70859db22 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.484579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f543239-08d1-47ca-b2e0-a75d5a3df53d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.492958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:25.518790 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea3f38-6ebd-4aed-9ec5-603e82a9a87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:25.519303 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea3f38-6ebd-4aed-9ec5-603e82a9a87b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.519585 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea3f38-6ebd-4aed-9ec5-603e82a9a87b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.519876 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea3f38-6ebd-4aed-9ec5-603e82a9a87b None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.520511 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea3f38-6ebd-4aed-9ec5-603e82a9a87b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:25.520809 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea3f38-6ebd-4aed-9ec5-603e82a9a87b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:25.522166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea3f38-6ebd-4aed-9ec5-603e82a9a87b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:25.538557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 412/810] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:25.539261 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea3f38-6ebd-4aed-9ec5-603e82a9a87b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.580911 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:25.582093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a971a1f-ffee-4755-b8ec-88f70859db22 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.615294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcec1b-54ea-4205-a3ec-133c25f1f4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:25.615930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcec1b-54ea-4205-a3ec-133c25f1f4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.616209 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcec1b-54ea-4205-a3ec-133c25f1f4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.616479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcec1b-54ea-4205-a3ec-133c25f1f4c7 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.617111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcec1b-54ea-4205-a3ec-133c25f1f4c7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:25.617414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcec1b-54ea-4205-a3ec-133c25f1f4c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:25.618775 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcec1b-54ea-4205-a3ec-133c25f1f4c7 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:25.625222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 399/811] 10.222.0.58 () {56 vars in 1124 bytes} [Wed Aug 24 12:20:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:25.625835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcec1b-54ea-4205-a3ec-133c25f1f4c7 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.640895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea3f38-6ebd-4aed-9ec5-603e82a9a87b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.676863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8220cd1-7f4c-4715-918d-ee7ce8b0d4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:25.677474 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8220cd1-7f4c-4715-918d-ee7ce8b0d4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.677764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8220cd1-7f4c-4715-918d-ee7ce8b0d4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.678034 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8220cd1-7f4c-4715-918d-ee7ce8b0d4d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.678674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8220cd1-7f4c-4715-918d-ee7ce8b0d4d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:25.678977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8220cd1-7f4c-4715-918d-ee7ce8b0d4d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:25.680380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8220cd1-7f4c-4715-918d-ee7ce8b0d4d2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:25.692092 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 413/812] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:25.692875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8220cd1-7f4c-4715-918d-ee7ce8b0d4d2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.719204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:25.735828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcec1b-54ea-4205-a3ec-133c25f1f4c7 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.772004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc2db2-a2f5-4cda-84ea-60b602fbcc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:25.772531 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc2db2-a2f5-4cda-84ea-60b602fbcc39 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.772810 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc2db2-a2f5-4cda-84ea-60b602fbcc39 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.773090 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc2db2-a2f5-4cda-84ea-60b602fbcc39 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.773732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc2db2-a2f5-4cda-84ea-60b602fbcc39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:25.774039 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc2db2-a2f5-4cda-84ea-60b602fbcc39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:25.775474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc2db2-a2f5-4cda-84ea-60b602fbcc39 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:25.787175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 400/813] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:25.788232 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc2db2-a2f5-4cda-84ea-60b602fbcc39 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.788346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:25.794761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8220cd1-7f4c-4715-918d-ee7ce8b0d4d2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.828687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aabc67-8127-4cab-8878-1ac0a697219f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:25.829313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aabc67-8127-4cab-8878-1ac0a697219f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.829594 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aabc67-8127-4cab-8878-1ac0a697219f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.829868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aabc67-8127-4cab-8878-1ac0a697219f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.836516 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aabc67-8127-4cab-8878-1ac0a697219f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:25.838002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aabc67-8127-4cab-8878-1ac0a697219f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:25.848989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-03aabc67-8127-4cab-8878-1ac0a697219f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:25.848989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:25.848989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:25.848989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:25.848989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:25.848989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:25.848989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:25.848989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:25.848989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:25.848989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:25.848989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:25.848989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:25.848989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:25.848989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:25.848989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:25.849872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:25.850765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:25.850765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:25.850765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:25.850765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:25.850765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:25.850765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:25.850765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:25.850765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:25.850765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 414/814] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:25.851264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aabc67-8127-4cab-8878-1ac0a697219f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.898497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc2db2-a2f5-4cda-84ea-60b602fbcc39 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.932552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4698b-326b-47b0-abe3-50a7bced8065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:25.933185 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4698b-326b-47b0-abe3-50a7bced8065 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.933465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4698b-326b-47b0-abe3-50a7bced8065 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.933769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4698b-326b-47b0-abe3-50a7bced8065 None admin] PATH_INFO: `/v3/services/network` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.934354 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4698b-326b-47b0-abe3-50a7bced8065 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:25.935753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4698b-326b-47b0-abe3-50a7bced8065 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-89c4698b-326b-47b0-abe3-50a7bced8065 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:25.944493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:25.945724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:25.945724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:25.945724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:25.945724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:25.945724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:25.945724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:25.945724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:25.945724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:25.945724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:25.945724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:25.945724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:25.945724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:25.945724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:25.945724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:25.945724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:25.945724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:25.946914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:25.946914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:25.946914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:25.946914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:25.946914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:25.946914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:25.946914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:25.946914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:25.946914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:25.946914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:25.946914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:25.946914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:25.946914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 12:20:25.946914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:25.946914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 401/815] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:25 2022] GET /identity/v3/services/network => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:25.947977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4698b-326b-47b0-abe3-50a7bced8065 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:25.957859 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:25.958311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aabc67-8127-4cab-8878-1ac0a697219f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:25.993744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a2fc8-9154-426a-a1f3-cc386d32f06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:25.994367 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a2fc8-9154-426a-a1f3-cc386d32f06a None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:25.994647 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a2fc8-9154-426a-a1f3-cc386d32f06a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:25.994913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a2fc8-9154-426a-a1f3-cc386d32f06a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:25.995563 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a2fc8-9154-426a-a1f3-cc386d32f06a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:25.996959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a2fc8-9154-426a-a1f3-cc386d32f06a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.017379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 415/816] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:20:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:26.018184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a2fc8-9154-426a-a1f3-cc386d32f06a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:26.050241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4698b-326b-47b0-abe3-50a7bced8065 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:26.053486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:26.098905 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419c6be-8ca2-4435-a559-2e0ce8163910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.099904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419c6be-8ca2-4435-a559-2e0ce8163910 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.100334 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419c6be-8ca2-4435-a559-2e0ce8163910 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.100766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419c6be-8ca2-4435-a559-2e0ce8163910 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.109887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e419c6be-8ca2-4435-a559-2e0ce8163910 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:26.111369 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e419c6be-8ca2-4435-a559-2e0ce8163910 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.123519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-e419c6be-8ca2-4435-a559-2e0ce8163910 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:26.123519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:26.123519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:26.123519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:26.123519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:26.123519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:26.123519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:26.123519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:26.123519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:26.123519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:26.123519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:26.123519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:26.123519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:26.123519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:26.123519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:26.123519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:26.124574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:26.130359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:26.130359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:26.130359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:26.130359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:26.130359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:26.130359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:26.130359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:26.130359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:26.130359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:26.130359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:26.130359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 402/817] 10.222.0.58 () {56 vars in 1093 bytes} [Wed Aug 24 12:20:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:26.130359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419c6be-8ca2-4435-a559-2e0ce8163910 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:26.162149 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a2fc8-9154-426a-a1f3-cc386d32f06a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:26.197903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:26.208620 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f9d39-4c92-444b-bac3-610cfafa968c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.209269 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f9d39-4c92-444b-bac3-610cfafa968c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.209553 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f9d39-4c92-444b-bac3-610cfafa968c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.209833 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f9d39-4c92-444b-bac3-610cfafa968c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.217635 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f9d39-4c92-444b-bac3-610cfafa968c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:26.219103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f9d39-4c92-444b-bac3-610cfafa968c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.231293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-555f9d39-4c92-444b-bac3-610cfafa968c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:26.231293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:26.231293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:26.231293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:26.231293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:26.231293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:26.231293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:26.231293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:26.231293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:26.231293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:26.231293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:26.231293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:26.231293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:26.231293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:26.231293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:26.232235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:26.233273 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:26.233273 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:26.233273 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:26.233273 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:26.233273 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:26.233273 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:26.233273 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:26.233273 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:26.233273 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 416/818] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:26.233273 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f9d39-4c92-444b-bac3-610cfafa968c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:26.245553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419c6be-8ca2-4435-a559-2e0ce8163910 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:26.281993 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99770d90-f9cc-4733-ba32-5572ed185695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.282609 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99770d90-f9cc-4733-ba32-5572ed185695 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.282889 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99770d90-f9cc-4733-ba32-5572ed185695 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.283295 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99770d90-f9cc-4733-ba32-5572ed185695 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.284028 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99770d90-f9cc-4733-ba32-5572ed185695 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:26.284339 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99770d90-f9cc-4733-ba32-5572ed185695 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:26.285778 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99770d90-f9cc-4733-ba32-5572ed185695 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.299327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 403/819] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:26.300185 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99770d90-f9cc-4733-ba32-5572ed185695 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:26.321799 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:26.351067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f9d39-4c92-444b-bac3-610cfafa968c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:26.386230 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9ba09-a6df-419b-a36a-c7fdeaf9ffba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.386842 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9ba09-a6df-419b-a36a-c7fdeaf9ffba None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.387127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9ba09-a6df-419b-a36a-c7fdeaf9ffba None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.387404 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9ba09-a6df-419b-a36a-c7fdeaf9ffba None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.388080 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9ba09-a6df-419b-a36a-c7fdeaf9ffba None admin] RBAC: Adding query filter params (name=network) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:26.388386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9ba09-a6df-419b-a36a-c7fdeaf9ffba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:26.389766 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9ba09-a6df-419b-a36a-c7fdeaf9ffba None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.398799 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 417/820] 10.222.0.58 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:26.398799 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9ba09-a6df-419b-a36a-c7fdeaf9ffba None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:26.407796 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99770d90-f9cc-4733-ba32-5572ed185695 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:26.432629 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:26.445663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aef06-c619-417b-bf44-d8482995cd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.446265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aef06-c619-417b-bf44-d8482995cd8c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.446649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aef06-c619-417b-bf44-d8482995cd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.446815 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83aef06-c619-417b-bf44-d8482995cd8c None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.448352 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aef06-c619-417b-bf44-d8482995cd8c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:26.448646 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aef06-c619-417b-bf44-d8482995cd8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:26.450314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83aef06-c619-417b-bf44-d8482995cd8c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.469155 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 404/821] 10.222.0.58 () {56 vars in 1133 bytes} [Wed Aug 24 12:20:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:26.469979 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aef06-c619-417b-bf44-d8482995cd8c None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:26.512502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9ba09-a6df-419b-a36a-c7fdeaf9ffba None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:26.548320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa3343-fc08-45c9-a79f-0799cf6d8667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.549685 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa3343-fc08-45c9-a79f-0799cf6d8667 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.550151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa3343-fc08-45c9-a79f-0799cf6d8667 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.550609 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa3343-fc08-45c9-a79f-0799cf6d8667 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.551499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa3343-fc08-45c9-a79f-0799cf6d8667 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:26.552016 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa3343-fc08-45c9-a79f-0799cf6d8667 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:26.553669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa3343-fc08-45c9-a79f-0799cf6d8667 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.561197 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:26.572701 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 418/822] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:26.573646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa3343-fc08-45c9-a79f-0799cf6d8667 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:26.579445 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83aef06-c619-417b-bf44-d8482995cd8c None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:26.616324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e5425-f5ab-47c3-9be2-b3ba00156ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.616970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e5425-f5ab-47c3-9be2-b3ba00156ff8 None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.617242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e5425-f5ab-47c3-9be2-b3ba00156ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.617515 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e5425-f5ab-47c3-9be2-b3ba00156ff8 None admin] PATH_INFO: `/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/67756e31aeee48a897cfa8bd35a10119/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.619367 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e5425-f5ab-47c3-9be2-b3ba00156ff8 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user_id=67756e31aeee48a897cfa8bd35a10119, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:26.620831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e5425-f5ab-47c3-9be2-b3ba00156ff8 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.633756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 405/823] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:26 2022] PUT /identity/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/67756e31aeee48a897cfa8bd35a10119/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:20:26.635302 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e5425-f5ab-47c3-9be2-b3ba00156ff8 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:26.696336 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa3343-fc08-45c9-a79f-0799cf6d8667 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:26.703563 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:26.737660 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a886b35-a5cf-404e-a436-28ace676de1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.738286 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a886b35-a5cf-404e-a436-28ace676de1e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.738562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a886b35-a5cf-404e-a436-28ace676de1e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.738837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a886b35-a5cf-404e-a436-28ace676de1e None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.739472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a886b35-a5cf-404e-a436-28ace676de1e None admin] RBAC: Adding query filter params (type=network) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:26.739819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a886b35-a5cf-404e-a436-28ace676de1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:26.741213 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a886b35-a5cf-404e-a436-28ace676de1e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.749326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 419/824] 10.222.0.58 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:26.750152 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a886b35-a5cf-404e-a436-28ace676de1e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:26.800475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e5425-f5ab-47c3-9be2-b3ba00156ff8 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:26.835044 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:26.846859 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0a254-6d17-4d7f-bb28-535e41cced0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.847715 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0a254-6d17-4d7f-bb28-535e41cced0d None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.848137 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0a254-6d17-4d7f-bb28-535e41cced0d None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.848585 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0a254-6d17-4d7f-bb28-535e41cced0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.890011 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a886b35-a5cf-404e-a436-28ace676de1e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:26.939720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c7fee-b26c-4d67-83b2-05d3d1718d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:26.941267 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c7fee-b26c-4d67-83b2-05d3d1718d92 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:26.941773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c7fee-b26c-4d67-83b2-05d3d1718d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:26.942198 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c7fee-b26c-4d67-83b2-05d3d1718d92 None admin] PATH_INFO: `/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/adc6a2b3f6bd45178cc7ffb181c3c016/roles/5a12926bd458420e81e72ecf825d67ae` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:26.947786 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0a254-6d17-4d7f-bb28-535e41cced0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:26.949413 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae0a254-6d17-4d7f-bb28-535e41cced0d None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.963280 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c7fee-b26c-4d67-83b2-05d3d1718d92 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user_id=adc6a2b3f6bd45178cc7ffb181c3c016, role_id=5a12926bd458420e81e72ecf825d67ae)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:26.965830 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c7fee-b26c-4d67-83b2-05d3d1718d92 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:26.984796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 420/825] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:26 2022] PUT /identity/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/adc6a2b3f6bd45178cc7ffb181c3c016/roles/5a12926bd458420e81e72ecf825d67ae => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:20:26.985967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c7fee-b26c-4d67-83b2-05d3d1718d92 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:27.000311 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:27.129116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 406/826] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:20:27.134756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0a254-6d17-4d7f-bb28-535e41cced0d None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:27.184498 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c7fee-b26c-4d67-83b2-05d3d1718d92 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:27.201186 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:27.226050 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75d09a-b65b-436c-9321-69e8bb0dad66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:27.226914 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75d09a-b65b-436c-9321-69e8bb0dad66 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:27.227284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75d09a-b65b-436c-9321-69e8bb0dad66 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:27.227676 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75d09a-b65b-436c-9321-69e8bb0dad66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:27.229058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75d09a-b65b-436c-9321-69e8bb0dad66 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:27.230515 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75d09a-b65b-436c-9321-69e8bb0dad66 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:27.260307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 421/827] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:20:26 2022] POST /identity/v3/endpoints => generated 337 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:27.283365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0a254-6d17-4d7f-bb28-535e41cced0d None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:27.336763 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70965b-24a6-4814-9710-bdee4b3b78bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:27.337802 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70965b-24a6-4814-9710-bdee4b3b78bf None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:27.338263 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70965b-24a6-4814-9710-bdee4b3b78bf None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:27.338821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70965b-24a6-4814-9710-bdee4b3b78bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:27.349023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:27.354561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70965b-24a6-4814-9710-bdee4b3b78bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:27.361056 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d70965b-24a6-4814-9710-bdee4b3b78bf None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:27.377924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-3d70965b-24a6-4814-9710-bdee4b3b78bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:27.377924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:27.377924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:27.377924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:27.377924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:27.377924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:27.377924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:27.377924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:27.377924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:27.377924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:27.377924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:27.377924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:27.377924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:27.377924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:27.377924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:27.377924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:27.378961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:27.379948 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:27.379948 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:27.379948 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:27.381525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 407/828] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:27.387267 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75d09a-b65b-436c-9321-69e8bb0dad66 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:27.516951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75d09a-b65b-436c-9321-69e8bb0dad66 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:27.553842 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b86c2-ba3b-401c-a445-72bc4ce678f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:27.554601 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b86c2-ba3b-401c-a445-72bc4ce678f8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:27.554975 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b86c2-ba3b-401c-a445-72bc4ce678f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:27.555345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b86c2-ba3b-401c-a445-72bc4ce678f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:27.556124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b86c2-ba3b-401c-a445-72bc4ce678f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:27.556560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b86c2-ba3b-401c-a445-72bc4ce678f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:27.558026 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628b86c2-ba3b-401c-a445-72bc4ce678f8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:27.570529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 422/829] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:27.576025 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70965b-24a6-4814-9710-bdee4b3b78bf None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:27.683731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d70965b-24a6-4814-9710-bdee4b3b78bf None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:27.727027 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b08f4-4dc7-46e9-9d4d-a6881fbf5048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:27.727685 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b08f4-4dc7-46e9-9d4d-a6881fbf5048 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:27.727965 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b08f4-4dc7-46e9-9d4d-a6881fbf5048 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:27.728238 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b08f4-4dc7-46e9-9d4d-a6881fbf5048 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:27.736233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b08f4-4dc7-46e9-9d4d-a6881fbf5048 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:27.737677 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b08f4-4dc7-46e9-9d4d-a6881fbf5048 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:27.749091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-3f1b08f4-4dc7-46e9-9d4d-a6881fbf5048 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:27.749091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:27.749091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:27.749091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:27.749091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:27.749091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:27.749091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:27.749091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:27.749091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:27.749091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:27.749091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:27.749091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:27.749091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:27.749091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:27.749091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:27.750009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:27.751278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:27.751278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:27.751278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:27.751278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:27.751278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:27.751278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:27.751278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:27.751278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:27.751278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 408/830] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:27.755922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b86c2-ba3b-401c-a445-72bc4ce678f8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:27.787292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:27.870070 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b86c2-ba3b-401c-a445-72bc4ce678f8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:27.908304 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e8905-5c0f-4566-b1f4-7c9232415151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:27.908922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e8905-5c0f-4566-b1f4-7c9232415151 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:27.909205 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e8905-5c0f-4566-b1f4-7c9232415151 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:27.909479 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e8905-5c0f-4566-b1f4-7c9232415151 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:27.910178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e8905-5c0f-4566-b1f4-7c9232415151 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:27.910482 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e8905-5c0f-4566-b1f4-7c9232415151 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:27.911971 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e8905-5c0f-4566-b1f4-7c9232415151 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:27.924967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 423/831] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:27.930792 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b08f4-4dc7-46e9-9d4d-a6881fbf5048 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:27.994260 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:28.049944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b08f4-4dc7-46e9-9d4d-a6881fbf5048 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:28.055088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306d804-8c41-42fc-bc5d-29ca22bcf06f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.055381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306d804-8c41-42fc-bc5d-29ca22bcf06f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.055695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306d804-8c41-42fc-bc5d-29ca22bcf06f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.057449 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 424/832] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:28.062435 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13e854-f14f-45dc-86c8-4f8b93298f76 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.062717 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13e854-f14f-45dc-86c8-4f8b93298f76 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.062992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13e854-f14f-45dc-86c8-4f8b93298f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.084906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8c13e854-f14f-45dc-86c8-4f8b93298f76 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:28.085838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09490ccb-c9e3-4cb9-9eb2-3448bf2a0baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:28.086093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8c13e854-f14f-45dc-86c8-4f8b93298f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:28.086813 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09490ccb-c9e3-4cb9-9eb2-3448bf2a0baa None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.087475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09490ccb-c9e3-4cb9-9eb2-3448bf2a0baa None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.088543 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09490ccb-c9e3-4cb9-9eb2-3448bf2a0baa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.089373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09490ccb-c9e3-4cb9-9eb2-3448bf2a0baa None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=311dbed3ab0b44a7830a93227ff9ad33) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:28.089750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09490ccb-c9e3-4cb9-9eb2-3448bf2a0baa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:28.091169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09490ccb-c9e3-4cb9-9eb2-3448bf2a0baa None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:28.097546 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 409/833] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:27 2022] GET /identity/v3/role_assignments?user.id=311dbed3ab0b44a7830a93227ff9ad33&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:28.170775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 425/834] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:28.440034 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09490ccb-c9e3-4cb9-9eb2-3448bf2a0baa None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:28.546915 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09490ccb-c9e3-4cb9-9eb2-3448bf2a0baa None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:28.562421 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:28.584232 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d49c3a-1ff3-4da3-aced-78098a25b9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:28.585015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d49c3a-1ff3-4da3-aced-78098a25b9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.585397 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d49c3a-1ff3-4da3-aced-78098a25b9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.585770 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d49c3a-1ff3-4da3-aced-78098a25b9d3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.586458 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d49c3a-1ff3-4da3-aced-78098a25b9d3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:28.587976 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d49c3a-1ff3-4da3-aced-78098a25b9d3 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-92d49c3a-1ff3-4da3-aced-78098a25b9d3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:28.597217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:28.598439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:28.598439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:28.598439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:28.598439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:28.598439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:28.598439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:28.598439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:28.598439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:28.598439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:28.598439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:28.598439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:28.598439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:28.598439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:28.598439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:28.598439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:28.598439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:28.599497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:28.599497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:28.599497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:28.599497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:28.599497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:28.599497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:28.599497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:28.599497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:28.599497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:28.599497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:28.599497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:28.599497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:28.599497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:28.599497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:28.602353 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 410/835] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:28.608228 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13e854-f14f-45dc-86c8-4f8b93298f76 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:28.627915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:28.713901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13e854-f14f-45dc-86c8-4f8b93298f76 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:28.750667 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e90c4f-24d4-45cf-a55f-b91956aac35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:28.751446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e90c4f-24d4-45cf-a55f-b91956aac35d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.751853 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e90c4f-24d4-45cf-a55f-b91956aac35d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.752225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e90c4f-24d4-45cf-a55f-b91956aac35d None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.752962 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e90c4f-24d4-45cf-a55f-b91956aac35d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:28.753370 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e90c4f-24d4-45cf-a55f-b91956aac35d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:28.754874 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e90c4f-24d4-45cf-a55f-b91956aac35d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:28.760413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 426/836] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:28.768728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d49c3a-1ff3-4da3-aced-78098a25b9d3 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:28.833890 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74e816-c5e1-49bd-ad7f-1a47a57b6c74 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.834437 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74e816-c5e1-49bd-ad7f-1a47a57b6c74 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.834860 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74e816-c5e1-49bd-ad7f-1a47a57b6c74 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.836653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 427/837] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:28.841987 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e58586-90d7-49ec-a074-398bad1a2719 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.842443 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e58586-90d7-49ec-a074-398bad1a2719 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.842865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e58586-90d7-49ec-a074-398bad1a2719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.862452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-84e58586-90d7-49ec-a074-398bad1a2719 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:28.863773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-84e58586-90d7-49ec-a074-398bad1a2719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:28.884103 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d49c3a-1ff3-4da3-aced-78098a25b9d3 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:28.926266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1f9fc-9b1f-45ff-94ec-39f7f8cd5aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:28.927116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1f9fc-9b1f-45ff-94ec-39f7f8cd5aed None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:28.927495 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1f9fc-9b1f-45ff-94ec-39f7f8cd5aed None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:28.927888 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1f9fc-9b1f-45ff-94ec-39f7f8cd5aed None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:28.928622 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1f9fc-9b1f-45ff-94ec-39f7f8cd5aed None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:28.929043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1f9fc-9b1f-45ff-94ec-39f7f8cd5aed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:28.930771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1f9fc-9b1f-45ff-94ec-39f7f8cd5aed None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:28.936147 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 411/838] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:28.941958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 428/839] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:29.215268 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1f9fc-9b1f-45ff-94ec-39f7f8cd5aed None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:29.261874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:29.272710 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe7ca2-0837-4248-a95e-578ca754a08e None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:29.272998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe7ca2-0837-4248-a95e-578ca754a08e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:29.273276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe7ca2-0837-4248-a95e-578ca754a08e None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:29.274744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 429/840] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:29.281698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6fd87-85f8-4908-b573-b642001ab8c8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:29.281985 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6fd87-85f8-4908-b573-b642001ab8c8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:29.282261 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6fd87-85f8-4908-b573-b642001ab8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:29.333032 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1f9fc-9b1f-45ff-94ec-39f7f8cd5aed None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:29.370871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d39ba1-fdae-45a0-8452-19a89cd8e359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:29.371525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d39ba1-fdae-45a0-8452-19a89cd8e359 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:29.371838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d39ba1-fdae-45a0-8452-19a89cd8e359 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:29.372112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d39ba1-fdae-45a0-8452-19a89cd8e359 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:29.384004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d39ba1-fdae-45a0-8452-19a89cd8e359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:29.389957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-84d39ba1-fdae-45a0-8452-19a89cd8e359 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:29.389957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:29.389957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:29.389957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:29.389957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:29.389957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:29.389957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:29.389957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:29.389957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:29.389957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:29.389957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:29.389957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:29.389957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:29.389957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:29.389957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:29.389957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:29.390960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:29.390960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:29.390960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:29.390960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:29.390960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:29.390960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:29.390960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:29.390960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:29.390960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:29.390960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:29.390960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:29.390960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:29.390960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:29.390960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:29.390960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:29.390960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:29.397121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:29.397121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:29.397121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:29.397121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:29.397121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:29.397121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:29.397121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:29.397121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:29.397121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:29.397121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:29.397121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:29.397121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:29.397121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 412/841] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:29.397848 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d39ba1-fdae-45a0-8452-19a89cd8e359 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:29.434123 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8fd6fd87-85f8-4908-b573-b642001ab8c8 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:29.435343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8fd6fd87-85f8-4908-b573-b642001ab8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:29.445397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:29.515319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d39ba1-fdae-45a0-8452-19a89cd8e359 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:29.518769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:29.533154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 430/842] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:29.559356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02ecbd-0565-46c6-b76d-1e99dfc9c2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:29.560013 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02ecbd-0565-46c6-b76d-1e99dfc9c2cc None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:29.560295 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02ecbd-0565-46c6-b76d-1e99dfc9c2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:29.560570 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02ecbd-0565-46c6-b76d-1e99dfc9c2cc None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:29.561231 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02ecbd-0565-46c6-b76d-1e99dfc9c2cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:29.561536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02ecbd-0565-46c6-b76d-1e99dfc9c2cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:29.562989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02ecbd-0565-46c6-b76d-1e99dfc9c2cc None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:29.573213 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 413/843] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:29.579058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6fd87-85f8-4908-b573-b642001ab8c8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:29.684200 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6fd87-85f8-4908-b573-b642001ab8c8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:29.720111 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c86a5-51ff-4755-a76c-0b31f5054180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:29.720792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c86a5-51ff-4755-a76c-0b31f5054180 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:29.721067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c86a5-51ff-4755-a76c-0b31f5054180 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:29.721435 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c86a5-51ff-4755-a76c-0b31f5054180 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:29.732032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c86a5-51ff-4755-a76c-0b31f5054180 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:29.733474 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c86a5-51ff-4755-a76c-0b31f5054180 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:29.735281 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:29.749513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-ee9c86a5-51ff-4755-a76c-0b31f5054180 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:29.749513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:29.749513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:29.749513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:29.749513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:29.749513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:29.749513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:29.749513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:29.749513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:29.749513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:29.749513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:29.749513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:29.749513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:29.749513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:29.749513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:29.749513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:29.750495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:29.751422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:29.751422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:29.751422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:29.751422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 431/844] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:29.757159 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02ecbd-0565-46c6-b76d-1e99dfc9c2cc None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:29.866196 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02ecbd-0565-46c6-b76d-1e99dfc9c2cc None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:29.877991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c86a5-51ff-4755-a76c-0b31f5054180 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:29.906727 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdfd12-cac3-470b-9c0d-6babfa51cffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:29.907662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdfd12-cac3-470b-9c0d-6babfa51cffd None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:29.907943 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdfd12-cac3-470b-9c0d-6babfa51cffd None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:29.908217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdfd12-cac3-470b-9c0d-6babfa51cffd None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:29.908858 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fdfd12-cac3-470b-9c0d-6babfa51cffd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:29.909158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fdfd12-cac3-470b-9c0d-6babfa51cffd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:29.910525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fdfd12-cac3-470b-9c0d-6babfa51cffd None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:29.927664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 414/845] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:29.932609 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdfd12-cac3-470b-9c0d-6babfa51cffd None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:29.969088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:30.002369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c86a5-51ff-4755-a76c-0b31f5054180 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:30.052885 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005ce62-24b4-4e3f-94b3-8a0c6f932c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:30.053827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005ce62-24b4-4e3f-94b3-8a0c6f932c2e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.054119 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005ce62-24b4-4e3f-94b3-8a0c6f932c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.054395 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005ce62-24b4-4e3f-94b3-8a0c6f932c2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.060439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdfd12-cac3-470b-9c0d-6babfa51cffd None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:30.071296 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005ce62-24b4-4e3f-94b3-8a0c6f932c2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:30.077586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e005ce62-24b4-4e3f-94b3-8a0c6f932c2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:30.077586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:30.077586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:30.077586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:30.077586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:30.077586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:30.077586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:30.077586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:30.077586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:30.077586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:30.077586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:30.077586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:30.077586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:30.077586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:30.077586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:30.077586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:30.078555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:30.078555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:30.078555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:30.078555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:30.078555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:30.078555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:30.078555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:30.078555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:30.078555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:30.078555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:30.078555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:30.078555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:30.078555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:30.078555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:30.078555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:30.078555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:30.079414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:30.079414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:30.079414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:30.079414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:30.079414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:30.079414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:30.079414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:30.079414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:30.079414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:30.079414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:30.079414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:20:30.079414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:30.079414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 432/846] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:20:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:30.080178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb90df9-12af-463b-ba32-5575c564c794 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.080247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb90df9-12af-463b-ba32-5575c564c794 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.080986 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb90df9-12af-463b-ba32-5575c564c794 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.081945 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 433/847] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:30.084766 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb90df9-12af-463b-ba32-5575c564c794 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:30.108334 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3831093-3543-4ad1-9256-f282178853ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:30.108968 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3831093-3543-4ad1-9256-f282178853ff None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.109244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3831093-3543-4ad1-9256-f282178853ff None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.109511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3831093-3543-4ad1-9256-f282178853ff None admin] PATH_INFO: `/v3/users/swift` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.117238 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3831093-3543-4ad1-9256-f282178853ff None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:30.118645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3831093-3543-4ad1-9256-f282178853ff None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:30.130068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-f3831093-3543-4ad1-9256-f282178853ff None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:30.130068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:30.130068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:30.130068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:30.130068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:30.130068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:30.130068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:30.130068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:30.130068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:30.130068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:30.130068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:30.130068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:30.130068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:30.130068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:30.130068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:30.130068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:30.131030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:30.132038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:30.132038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:30.132038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:30.132038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:30.132038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:30.132038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:30.132038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:30.132038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:30.132038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 12:20:30.132038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:30.135499 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 415/848] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:30.137237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aee453-0bcb-4181-abcf-21144a5a942c None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.137527 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aee453-0bcb-4181-abcf-21144a5a942c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.137798 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aee453-0bcb-4181-abcf-21144a5a942c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.146774 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:30.157361 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-73aee453-0bcb-4181-abcf-21144a5a942c None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:30.158548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-73aee453-0bcb-4181-abcf-21144a5a942c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:30.196559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb90df9-12af-463b-ba32-5575c564c794 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:30.232697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 416/849] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:30.233571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aee453-0bcb-4181-abcf-21144a5a942c None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:30.240452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea79802-114d-408e-8ad2-4662737659e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:30.241331 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea79802-114d-408e-8ad2-4662737659e3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.241792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea79802-114d-408e-8ad2-4662737659e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.242240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea79802-114d-408e-8ad2-4662737659e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.243098 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea79802-114d-408e-8ad2-4662737659e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:30.243577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea79802-114d-408e-8ad2-4662737659e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:30.245376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea79802-114d-408e-8ad2-4662737659e3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:30.254910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:30.257739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 434/850] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:20:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:30.264869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea79802-114d-408e-8ad2-4662737659e3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:30.364175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aee453-0bcb-4181-abcf-21144a5a942c None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:30.403900 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d45e45-bf82-4112-940f-1681ad8b3676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:30.404505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d45e45-bf82-4112-940f-1681ad8b3676 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.404846 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d45e45-bf82-4112-940f-1681ad8b3676 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.405177 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d45e45-bf82-4112-940f-1681ad8b3676 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.405970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d45e45-bf82-4112-940f-1681ad8b3676 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:30.406493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d45e45-bf82-4112-940f-1681ad8b3676 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:30.408164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d45e45-bf82-4112-940f-1681ad8b3676 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:30.422108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea79802-114d-408e-8ad2-4662737659e3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:30.428072 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 417/851] 10.222.0.58 () {56 vars in 1135 bytes} [Wed Aug 24 12:20:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:30.434862 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d45e45-bf82-4112-940f-1681ad8b3676 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:30.451470 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:30.481076 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457256e-056f-4661-ba57-7f92888b22ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:30.482012 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457256e-056f-4661-ba57-7f92888b22ff None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.482359 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457256e-056f-4661-ba57-7f92888b22ff None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.482972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457256e-056f-4661-ba57-7f92888b22ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.502862 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457256e-056f-4661-ba57-7f92888b22ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:30.504827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457256e-056f-4661-ba57-7f92888b22ff None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:30.527176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-0457256e-056f-4661-ba57-7f92888b22ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:30.527176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:30.527176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:30.527176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:30.527176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:30.527176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:30.527176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:30.527176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:30.527176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:30.527176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:30.527176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:30.527176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:30.527176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:30.527176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:30.527176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:30.527176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:30.528302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:30.529231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:30.529231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:30.529231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:30.531789 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 435/852] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:30.535801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457256e-056f-4661-ba57-7f92888b22ff None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:30.538321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:30.580992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d45e45-bf82-4112-940f-1681ad8b3676 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:30.627751 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666179c-5a52-434c-9bef-4ca94873e5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:30.628825 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666179c-5a52-434c-9bef-4ca94873e5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.629301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666179c-5a52-434c-9bef-4ca94873e5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.629764 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666179c-5a52-434c-9bef-4ca94873e5c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.663426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457256e-056f-4661-ba57-7f92888b22ff None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:30.708534 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751b157-7521-472c-8c71-f32ed37a6d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:30.709204 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751b157-7521-472c-8c71-f32ed37a6d19 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.709420 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751b157-7521-472c-8c71-f32ed37a6d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.709695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751b157-7521-472c-8c71-f32ed37a6d19 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.710367 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751b157-7521-472c-8c71-f32ed37a6d19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:30.710670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751b157-7521-472c-8c71-f32ed37a6d19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:30.712156 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751b157-7521-472c-8c71-f32ed37a6d19 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:30.714992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666179c-5a52-434c-9bef-4ca94873e5c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:30.716480 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666179c-5a52-434c-9bef-4ca94873e5c1 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:30.725818 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 436/853] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:30.726559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751b157-7521-472c-8c71-f32ed37a6d19 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:30.762319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:30.807068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:30.855347 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751b157-7521-472c-8c71-f32ed37a6d19 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:30.855545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 418/854] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:30 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:20:30.856045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666179c-5a52-434c-9bef-4ca94873e5c1 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:30.905839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc94216-bda5-40d4-98bf-a5e077c7cb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:30.906482 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc94216-bda5-40d4-98bf-a5e077c7cb02 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:30.906766 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc94216-bda5-40d4-98bf-a5e077c7cb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:30.907042 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc94216-bda5-40d4-98bf-a5e077c7cb02 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:30.907654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc94216-bda5-40d4-98bf-a5e077c7cb02 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:30.909349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc94216-bda5-40d4-98bf-a5e077c7cb02 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:30.921108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-2fc94216-bda5-40d4-98bf-a5e077c7cb02 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:30.921108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:30.921108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:30.921108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:30.921108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:30.921108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:30.921108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:30.921108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:30.921108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:30.921108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:30.921108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:30.921108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:30.921108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:30.921108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:30.921108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:30.921108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:30.923839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:30.924983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:30.924983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:30.924983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:30.924983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:30.924983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:30.924983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:30.924983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:30.924983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:30.924983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:30.924983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:30.924983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:30.924983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:30.924983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:30.924983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:30.924983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 437/855] 10.222.0.58 () {56 vars in 1132 bytes} [Wed Aug 24 12:20:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:30.925996 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc94216-bda5-40d4-98bf-a5e077c7cb02 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:30.996399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666179c-5a52-434c-9bef-4ca94873e5c1 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:31.038608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bde5b-0e73-4119-997e-3f907506a1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:31.039247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bde5b-0e73-4119-997e-3f907506a1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.039526 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bde5b-0e73-4119-997e-3f907506a1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.039828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236bde5b-0e73-4119-997e-3f907506a1c1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.040411 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:31.048359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236bde5b-0e73-4119-997e-3f907506a1c1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:31.049817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236bde5b-0e73-4119-997e-3f907506a1c1 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:31.050483 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc94216-bda5-40d4-98bf-a5e077c7cb02 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:31.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-236bde5b-0e73-4119-997e-3f907506a1c1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:31.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:31.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:31.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:31.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:31.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:31.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:31.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:31.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:31.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:31.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:31.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:31.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:31.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:31.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:31.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:31.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:31.066710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:31.066710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:31.066710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:31.066710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:31.066710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:31.066710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:31.066710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:31.066710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:31.066710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:31.066710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:31.067791 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 419/856] 10.222.0.58 () {56 vars in 1117 bytes} [Wed Aug 24 12:20:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:31.069463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bde5b-0e73-4119-997e-3f907506a1c1 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:31.092310 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50940e2-7fc9-4226-929e-f71ad40b92e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:31.093038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50940e2-7fc9-4226-929e-f71ad40b92e2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.093384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50940e2-7fc9-4226-929e-f71ad40b92e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.093725 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50940e2-7fc9-4226-929e-f71ad40b92e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.105536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50940e2-7fc9-4226-929e-f71ad40b92e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:31.106340 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:31.107089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50940e2-7fc9-4226-929e-f71ad40b92e2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:31.123951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d50940e2-7fc9-4226-929e-f71ad40b92e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:31.123951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:31.123951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:31.123951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:31.123951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:31.123951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:31.123951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:31.123951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:31.123951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:31.123951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:31.123951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:31.123951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:31.123951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:31.123951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:31.123951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:31.123951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:31.125187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:31.126302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:31.126302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:31.126302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:31.126302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 438/857] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:31.126302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee7b0b-1c14-4cf2-8cb7-c63c3ff08e89 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.126666 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee7b0b-1c14-4cf2-8cb7-c63c3ff08e89 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.126833 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee7b0b-1c14-4cf2-8cb7-c63c3ff08e89 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.132000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 439/858] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:31.133328 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee7b0b-1c14-4cf2-8cb7-c63c3ff08e89 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:31.189145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236bde5b-0e73-4119-997e-3f907506a1c1 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:31.227208 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f462e3-8853-4bbf-b0be-bb6004fac4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:31.227870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f462e3-8853-4bbf-b0be-bb6004fac4dc None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.228218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f462e3-8853-4bbf-b0be-bb6004fac4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.229126 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f462e3-8853-4bbf-b0be-bb6004fac4dc None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.229766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f462e3-8853-4bbf-b0be-bb6004fac4dc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:31.230079 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f462e3-8853-4bbf-b0be-bb6004fac4dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:31.232153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f462e3-8853-4bbf-b0be-bb6004fac4dc None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:31.239347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 420/859] 10.222.0.58 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:31.240576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f462e3-8853-4bbf-b0be-bb6004fac4dc None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:31.259238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee7b0b-1c14-4cf2-8cb7-c63c3ff08e89 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:31.307865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac12fbf-0820-4b57-aadd-8f98b1559996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:31.308466 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac12fbf-0820-4b57-aadd-8f98b1559996 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.308745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac12fbf-0820-4b57-aadd-8f98b1559996 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.309023 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac12fbf-0820-4b57-aadd-8f98b1559996 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.309774 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac12fbf-0820-4b57-aadd-8f98b1559996 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:31.310077 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac12fbf-0820-4b57-aadd-8f98b1559996 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:31.311499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac12fbf-0820-4b57-aadd-8f98b1559996 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:31.328985 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 440/860] 10.222.0.58 () {56 vars in 1149 bytes} [Wed Aug 24 12:20:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:31.330179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf8586-9e0c-4a22-ad7a-f30402a9448e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.330466 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf8586-9e0c-4a22-ad7a-f30402a9448e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.330741 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf8586-9e0c-4a22-ad7a-f30402a9448e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.336750 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:31.338534 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:31.345212 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f462e3-8853-4bbf-b0be-bb6004fac4dc None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:31.349716 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-36bf8586-9e0c-4a22-ad7a-f30402a9448e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:31.350883 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-36bf8586-9e0c-4a22-ad7a-f30402a9448e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:31.379109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f3527-da1d-4c2e-a213-9021d099a4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:31.379746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f3527-da1d-4c2e-a213-9021d099a4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.380030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f3527-da1d-4c2e-a213-9021d099a4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.380304 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f3527-da1d-4c2e-a213-9021d099a4d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.380950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f3527-da1d-4c2e-a213-9021d099a4d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:31.381252 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f3527-da1d-4c2e-a213-9021d099a4d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:31.382621 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f3527-da1d-4c2e-a213-9021d099a4d3 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:31.394366 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 421/861] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:31.395176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f3527-da1d-4c2e-a213-9021d099a4d3 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:31.417157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 441/862] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:31.418009 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf8586-9e0c-4a22-ad7a-f30402a9448e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:31.498817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f3527-da1d-4c2e-a213-9021d099a4d3 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:31.535772 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf8586-9e0c-4a22-ad7a-f30402a9448e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:31.538948 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65727c2-b01b-46d9-be4b-5c04894dccb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:31.539566 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65727c2-b01b-46d9-be4b-5c04894dccb1 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.539876 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65727c2-b01b-46d9-be4b-5c04894dccb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.540145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65727c2-b01b-46d9-be4b-5c04894dccb1 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.541327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65727c2-b01b-46d9-be4b-5c04894dccb1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:31.541631 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65727c2-b01b-46d9-be4b-5c04894dccb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:31.542994 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65727c2-b01b-46d9-be4b-5c04894dccb1 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:31.555906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 422/863] 10.222.0.58 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:31.555906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65727c2-b01b-46d9-be4b-5c04894dccb1 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:31.581261 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:31.587941 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b2059-ec5b-424d-87e9-f90f08986039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:31.589336 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b2059-ec5b-424d-87e9-f90f08986039 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.589608 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b2059-ec5b-424d-87e9-f90f08986039 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.590222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b2059-ec5b-424d-87e9-f90f08986039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.600143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:31.673703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b2059-ec5b-424d-87e9-f90f08986039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:31.675658 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701b2059-ec5b-424d-87e9-f90f08986039 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:31.704107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65727c2-b01b-46d9-be4b-5c04894dccb1 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:31.753365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaaeef-75a6-4782-96c8-227d2a7ca6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:31.753922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaaeef-75a6-4782-96c8-227d2a7ca6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.754204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaaeef-75a6-4782-96c8-227d2a7ca6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.754479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaaeef-75a6-4782-96c8-227d2a7ca6c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.762076 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadaaeef-75a6-4782-96c8-227d2a7ca6c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:31.763541 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadaaeef-75a6-4782-96c8-227d2a7ca6c3 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:31.774847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-eadaaeef-75a6-4782-96c8-227d2a7ca6c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:31.774847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:31.774847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:31.774847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:31.774847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:31.774847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:31.774847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:31.774847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:31.774847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:31.774847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:31.774847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:31.774847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:31.774847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:31.774847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:31.774847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:31.775833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:31.776747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:31.776747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:31.776747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:31.776747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:31.776747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:31.776747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:31.776747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:31.776747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:31.776747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 423/864] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:31.777243 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaaeef-75a6-4782-96c8-227d2a7ca6c3 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:31.793721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 442/865] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:31 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:20:31.793721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b2059-ec5b-424d-87e9-f90f08986039 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:31.825199 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:31.888671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:31.903250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaaeef-75a6-4782-96c8-227d2a7ca6c3 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:31.903906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b2059-ec5b-424d-87e9-f90f08986039 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:31.938672 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e2711-ff6c-41eb-bc1e-e5f4ac47c7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:31.938933 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78760cc-2617-48d6-9081-e28f8d412ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:31.939304 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e2711-ff6c-41eb-bc1e-e5f4ac47c7a1 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.939581 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e2711-ff6c-41eb-bc1e-e5f4ac47c7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.939821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78760cc-2617-48d6-9081-e28f8d412ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:31.939914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e2711-ff6c-41eb-bc1e-e5f4ac47c7a1 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.940175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78760cc-2617-48d6-9081-e28f8d412ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:31.940500 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e2711-ff6c-41eb-bc1e-e5f4ac47c7a1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:31.940593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78760cc-2617-48d6-9081-e28f8d412ed6 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:31.941270 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78760cc-2617-48d6-9081-e28f8d412ed6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:31.941655 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78760cc-2617-48d6-9081-e28f8d412ed6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:31.941868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e2711-ff6c-41eb-bc1e-e5f4ac47c7a1 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:31.943205 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78760cc-2617-48d6-9081-e28f8d412ed6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:31.957085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 443/866] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:31.957907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78760cc-2617-48d6-9081-e28f8d412ed6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:31.967758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 424/867] 10.222.0.58 () {60 vars in 1141 bytes} [Wed Aug 24 12:20:31 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:31.967758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e2711-ff6c-41eb-bc1e-e5f4ac47c7a1 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:32.076812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:32.088241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78760cc-2617-48d6-9081-e28f8d412ed6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:32.140500 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e2711-ff6c-41eb-bc1e-e5f4ac47c7a1 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:32.149337 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882abff-f457-41f9-b922-b4134d4ef512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:32.150499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882abff-f457-41f9-b922-b4134d4ef512 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:32.150847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882abff-f457-41f9-b922-b4134d4ef512 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:32.151123 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882abff-f457-41f9-b922-b4134d4ef512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:32.164983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882abff-f457-41f9-b922-b4134d4ef512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:32.167431 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882abff-f457-41f9-b922-b4134d4ef512 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:32.182322 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84c2b6-2aba-46da-b880-c20e1d0c1ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:32.183305 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84c2b6-2aba-46da-b880-c20e1d0c1ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:32.183849 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84c2b6-2aba-46da-b880-c20e1d0c1ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:32.184046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e882abff-f457-41f9-b922-b4134d4ef512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:32.184046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:32.184046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:32.184046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:32.184046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:32.184046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:32.184046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:32.184046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:32.184046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:32.184046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:32.184046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:32.184046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:32.184046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:32.184046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:32.184046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:32.184046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:32.184969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:32.186293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:32.186293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:32.186293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:32.186293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 444/868] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:32.187014 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84c2b6-2aba-46da-b880-c20e1d0c1ee3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:32.187988 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d84c2b6-2aba-46da-b880-c20e1d0c1ee3 None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=67756e31aeee48a897cfa8bd35a10119) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:32.188535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d84c2b6-2aba-46da-b880-c20e1d0c1ee3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:32.190333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d84c2b6-2aba-46da-b880-c20e1d0c1ee3 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:32.191439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882abff-f457-41f9-b922-b4134d4ef512 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:32.194227 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:32.199464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 425/869] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:31 2022] GET /identity/v3/role_assignments?user.id=67756e31aeee48a897cfa8bd35a10119&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:32.314786 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882abff-f457-41f9-b922-b4134d4ef512 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:32.353124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da324054-5e51-4c21-aaad-427b46a8c861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:32.353663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da324054-5e51-4c21-aaad-427b46a8c861 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:32.353940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da324054-5e51-4c21-aaad-427b46a8c861 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:32.354205 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da324054-5e51-4c21-aaad-427b46a8c861 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:32.354858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da324054-5e51-4c21-aaad-427b46a8c861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:32.355160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da324054-5e51-4c21-aaad-427b46a8c861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:32.356562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da324054-5e51-4c21-aaad-427b46a8c861 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:32.369217 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 445/870] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:32.374622 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84c2b6-2aba-46da-b880-c20e1d0c1ee3 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:32.483003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84c2b6-2aba-46da-b880-c20e1d0c1ee3 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:32.519292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4e85c-8de1-4085-894e-7462916c8c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:32.519959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4e85c-8de1-4085-894e-7462916c8c09 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:32.520240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4e85c-8de1-4085-894e-7462916c8c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:32.520516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe4e85c-8de1-4085-894e-7462916c8c09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:32.528358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe4e85c-8de1-4085-894e-7462916c8c09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:32.529807 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe4e85c-8de1-4085-894e-7462916c8c09 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:32.541861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-cfe4e85c-8de1-4085-894e-7462916c8c09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:32.541861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:32.541861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:32.541861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:32.541861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:32.541861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:32.541861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:32.541861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:32.541861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:32.541861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:32.541861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:32.541861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:32.541861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:32.541861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:32.541861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:32.542797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:32.543743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:32.543743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:32.543743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:32.543743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:32.543743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:32.543743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:32.543743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:32.543743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:32.543743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 426/871] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:32.550408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da324054-5e51-4c21-aaad-427b46a8c861 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:32.555191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:32.667716 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da324054-5e51-4c21-aaad-427b46a8c861 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:32.705407 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6952821-8942-4059-bfa6-f0925c5fce1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:32.706017 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6952821-8942-4059-bfa6-f0925c5fce1b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:32.706297 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6952821-8942-4059-bfa6-f0925c5fce1b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:32.706574 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6952821-8942-4059-bfa6-f0925c5fce1b None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:32.707279 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6952821-8942-4059-bfa6-f0925c5fce1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:32.707588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6952821-8942-4059-bfa6-f0925c5fce1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:32.709091 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6952821-8942-4059-bfa6-f0925c5fce1b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:32.721762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 446/872] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:32.727351 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4e85c-8de1-4085-894e-7462916c8c09 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:32.789995 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:32.834558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe4e85c-8de1-4085-894e-7462916c8c09 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:32.870070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc398ba-1f57-4b88-9772-75cea23e8f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:32.870925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc398ba-1f57-4b88-9772-75cea23e8f6a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:32.871208 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc398ba-1f57-4b88-9772-75cea23e8f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:32.871482 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc398ba-1f57-4b88-9772-75cea23e8f6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:32.872277 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc398ba-1f57-4b88-9772-75cea23e8f6a None admin] RBAC: Adding query filter params (role.id=5a12926bd458420e81e72ecf825d67ae, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=adc6a2b3f6bd45178cc7ffb181c3c016) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:32.872590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc398ba-1f57-4b88-9772-75cea23e8f6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:32.876913 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc398ba-1f57-4b88-9772-75cea23e8f6a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:32.883610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 427/873] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:32 2022] GET /identity/v3/role_assignments?user.id=adc6a2b3f6bd45178cc7ffb181c3c016&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=5a12926bd458420e81e72ecf825d67ae => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:33.520253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067201c8-3d30-44f8-9aff-598bb5ae221b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:33.520777 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067201c8-3d30-44f8-9aff-598bb5ae221b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:33.520777 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067201c8-3d30-44f8-9aff-598bb5ae221b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:33.522502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 447/874] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:33.527920 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04585e0-9e4a-4bca-9084-5eba0784e23d None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:33.528409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04585e0-9e4a-4bca-9084-5eba0784e23d None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:33.528489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04585e0-9e4a-4bca-9084-5eba0784e23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:33.546675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-f04585e0-9e4a-4bca-9084-5eba0784e23d None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:33.547902 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f04585e0-9e4a-4bca-9084-5eba0784e23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:33.621424 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 428/875] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:33 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:33.892873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067201c8-3d30-44f8-9aff-598bb5ae221b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:33.959085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f481cf-7aac-4b6c-b1df-20fa7774a472 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:33.959397 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f481cf-7aac-4b6c-b1df-20fa7774a472 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:33.959651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f481cf-7aac-4b6c-b1df-20fa7774a472 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:33.961049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:33.961232 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 429/876] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:33.966834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e47db-7c01-4dea-9ba8-47c915fbb61d None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:33.967112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e47db-7c01-4dea-9ba8-47c915fbb61d None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:33.967392 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401e47db-7c01-4dea-9ba8-47c915fbb61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:33.992270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-401e47db-7c01-4dea-9ba8-47c915fbb61d None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:33.993470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-401e47db-7c01-4dea-9ba8-47c915fbb61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:34.016712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067201c8-3d30-44f8-9aff-598bb5ae221b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:34.047022 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:34.055999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aceaae-85f0-4086-98bf-19deb5db962c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:34.057790 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aceaae-85f0-4086-98bf-19deb5db962c None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.058305 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aceaae-85f0-4086-98bf-19deb5db962c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.058774 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aceaae-85f0-4086-98bf-19deb5db962c None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.059753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aceaae-85f0-4086-98bf-19deb5db962c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:34.061432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aceaae-85f0-4086-98bf-19deb5db962c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:34.071670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 430/877] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:33 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:34.092347 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 448/878] 10.222.0.58 () {60 vars in 1141 bytes} [Wed Aug 24 12:20:33 2022] POST /identity/v3/services => generated 269 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:34.297374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30086874-bee8-4e0b-98c5-4bfec20aa5bf None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.297554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30086874-bee8-4e0b-98c5-4bfec20aa5bf None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.297835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30086874-bee8-4e0b-98c5-4bfec20aa5bf None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.299451 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 431/879] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:34.305074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca2b4d-512c-4a93-ab2f-821b75b73d15 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.305368 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca2b4d-512c-4a93-ab2f-821b75b73d15 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.305648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca2b4d-512c-4a93-ab2f-821b75b73d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.326134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-52ca2b4d-512c-4a93-ab2f-821b75b73d15 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:34.327331 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-52ca2b4d-512c-4a93-ab2f-821b75b73d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:34.346944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30086874-bee8-4e0b-98c5-4bfec20aa5bf None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:34.406707 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 449/880] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:34.460507 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30086874-bee8-4e0b-98c5-4bfec20aa5bf None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:34.501225 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085a64c-1e90-4b25-9582-72134e62efb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:34.502137 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085a64c-1e90-4b25-9582-72134e62efb7 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.502566 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085a64c-1e90-4b25-9582-72134e62efb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.502998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085a64c-1e90-4b25-9582-72134e62efb7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.503766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085a64c-1e90-4b25-9582-72134e62efb7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:34.505292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085a64c-1e90-4b25-9582-72134e62efb7 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-2085a64c-1e90-4b25-9582-72134e62efb7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:34.515009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:34.516401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:34.516401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:34.516401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:34.516401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:34.516401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:34.516401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:34.516401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:34.516401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:34.516401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:34.516401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:34.516401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:34.516401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:34.516401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:34.516401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:34.516401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:34.516401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:34.517491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:34.517491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:34.517491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:34.517491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:34.517491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:34.517491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:34.517491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:34.517491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:34.517491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:34.517491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:34.517491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:34.517491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:34.517491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:34.517491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:34.519990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 432/881] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:34.527668 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca2b4d-512c-4a93-ab2f-821b75b73d15 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:34.579476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:34.631830 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca2b4d-512c-4a93-ab2f-821b75b73d15 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:34.667573 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8ddab-c3a8-411a-bc0b-e86293dbb01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:34.668221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8ddab-c3a8-411a-bc0b-e86293dbb01f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.668498 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8ddab-c3a8-411a-bc0b-e86293dbb01f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.668768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8ddab-c3a8-411a-bc0b-e86293dbb01f None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.669401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8ddab-c3a8-411a-bc0b-e86293dbb01f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:34.669699 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8ddab-c3a8-411a-bc0b-e86293dbb01f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:34.671069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8ddab-c3a8-411a-bc0b-e86293dbb01f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:34.676345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 450/882] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:34.682448 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085a64c-1e90-4b25-9582-72134e62efb7 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:34.692294 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8ddab-c3a8-411a-bc0b-e86293dbb01f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:34.745470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:34.807997 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8ddab-c3a8-411a-bc0b-e86293dbb01f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:34.813205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085a64c-1e90-4b25-9582-72134e62efb7 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:34.823911 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:34.857664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72601384-5c83-48b5-8ea0-001e370bef88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:34.857867 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07ae16-9e83-461e-a9f6-dd4511b72c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:34.858213 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72601384-5c83-48b5-8ea0-001e370bef88 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.858325 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07ae16-9e83-461e-a9f6-dd4511b72c8f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:34.858483 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72601384-5c83-48b5-8ea0-001e370bef88 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.858579 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07ae16-9e83-461e-a9f6-dd4511b72c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:34.858753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72601384-5c83-48b5-8ea0-001e370bef88 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.858848 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07ae16-9e83-461e-a9f6-dd4511b72c8f None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:34.859324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72601384-5c83-48b5-8ea0-001e370bef88 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:34.859431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07ae16-9e83-461e-a9f6-dd4511b72c8f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:34.859739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07ae16-9e83-461e-a9f6-dd4511b72c8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:34.860780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72601384-5c83-48b5-8ea0-001e370bef88 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:34.861103 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07ae16-9e83-461e-a9f6-dd4511b72c8f None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:34.867765 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 433/883] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-72601384-5c83-48b5-8ea0-001e370bef88 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:34.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:34.874626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:34.874626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:34.874626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:34.874626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:34.874626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:34.874626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:34.874626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:34.874626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:34.874626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:34.874626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:34.874626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:34.874626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:34.874626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:34.874626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:34.874626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:34.874626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:34.875464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:34.875464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:34.875464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:34.875464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:34.875464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:34.875464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:34.875464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:34.875464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:34.875464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:34.875464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:34.875464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:34.875464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:34.875464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:34.875464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:34.875464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 451/884] 10.222.0.58 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:34.881461 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72601384-5c83-48b5-8ea0-001e370bef88 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:34.885152 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07ae16-9e83-461e-a9f6-dd4511b72c8f None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:34.991456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07ae16-9e83-461e-a9f6-dd4511b72c8f None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:34.999535 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72601384-5c83-48b5-8ea0-001e370bef88 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:35.006905 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:35.030718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5cf459-6d10-4055-93f1-d1fe35c78844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.031739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cf459-6d10-4055-93f1-d1fe35c78844 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.032222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cf459-6d10-4055-93f1-d1fe35c78844 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.032703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5cf459-6d10-4055-93f1-d1fe35c78844 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.033586 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5cf459-6d10-4055-93f1-d1fe35c78844 None admin] RBAC: Adding query filter params (interface=public, service_id=1cafa5b2751b4a30be407fc4d6ca5891, region_id=RegionOne) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:35.034070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5cf459-6d10-4055-93f1-d1fe35c78844 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.035823 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5cf459-6d10-4055-93f1-d1fe35c78844 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.038173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ac7d6-3cf6-4b2c-9a58-6581beb1adf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.038866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ac7d6-3cf6-4b2c-9a58-6581beb1adf5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.039041 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ac7d6-3cf6-4b2c-9a58-6581beb1adf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.039311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ac7d6-3cf6-4b2c-9a58-6581beb1adf5 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.039960 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ac7d6-3cf6-4b2c-9a58-6581beb1adf5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:35.040278 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ac7d6-3cf6-4b2c-9a58-6581beb1adf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.041640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ac7d6-3cf6-4b2c-9a58-6581beb1adf5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.042800 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 434/885] 10.222.0.58 () {56 vars in 1251 bytes} [Wed Aug 24 12:20:34 2022] GET /identity/v3/endpoints?service_id=1cafa5b2751b4a30be407fc4d6ca5891&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:35.044144 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be0e5d-0a03-41da-907e-5fb238ebfbcf None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.044574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be0e5d-0a03-41da-907e-5fb238ebfbcf None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.045010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be0e5d-0a03-41da-907e-5fb238ebfbcf None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.046360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 435/886] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:35.047789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be0e5d-0a03-41da-907e-5fb238ebfbcf None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:35.051198 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 452/887] 10.222.0.58 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:35.052404 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042805c-49e5-433d-b51f-fc42ae49c141 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.052689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042805c-49e5-433d-b51f-fc42ae49c141 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.052962 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042805c-49e5-433d-b51f-fc42ae49c141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.071043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e042805c-49e5-433d-b51f-fc42ae49c141 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:35.072911 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e042805c-49e5-433d-b51f-fc42ae49c141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:35.113675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:35.149223 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 453/888] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:35.150245 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042805c-49e5-433d-b51f-fc42ae49c141 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:35.168418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be0e5d-0a03-41da-907e-5fb238ebfbcf None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:35.217010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbf384-55c7-40bd-8ca2-1eb514347c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.218554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbf384-55c7-40bd-8ca2-1eb514347c28 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.219112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbf384-55c7-40bd-8ca2-1eb514347c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.219671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbf384-55c7-40bd-8ca2-1eb514347c28 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.220706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbf384-55c7-40bd-8ca2-1eb514347c28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.222861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbf384-55c7-40bd-8ca2-1eb514347c28 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.240294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 436/889] 10.222.0.58 () {56 vars in 1087 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/services => generated 1268 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:20:35.269612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042805c-49e5-433d-b51f-fc42ae49c141 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:35.310714 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826aefd1-afc2-4ddb-a184-2c506d63273b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.311335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826aefd1-afc2-4ddb-a184-2c506d63273b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.311616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826aefd1-afc2-4ddb-a184-2c506d63273b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.311917 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826aefd1-afc2-4ddb-a184-2c506d63273b None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.312556 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826aefd1-afc2-4ddb-a184-2c506d63273b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:35.312860 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826aefd1-afc2-4ddb-a184-2c506d63273b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.314325 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826aefd1-afc2-4ddb-a184-2c506d63273b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.320106 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 454/890] 10.222.0.58 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:35.430812 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbf384-55c7-40bd-8ca2-1eb514347c28 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:35.477848 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:35.542686 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbf384-55c7-40bd-8ca2-1eb514347c28 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:35.578875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770e985-fd89-4323-b4b0-c523ad531ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.579517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770e985-fd89-4323-b4b0-c523ad531ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.579827 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770e985-fd89-4323-b4b0-c523ad531ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.580103 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770e985-fd89-4323-b4b0-c523ad531ed1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.591062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770e985-fd89-4323-b4b0-c523ad531ed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:35.596568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-f770e985-fd89-4323-b4b0-c523ad531ed1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:20:35.596568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:35.596568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:35.596568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:35.596568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:35.596568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:35.596568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:35.596568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:35.596568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:35.596568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:35.596568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:35.596568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:35.596568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:35.596568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:35.596568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:35.596568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:35.598997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:35.598997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:35.598997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:35.598997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:35.598997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:35.598997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:35.598997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:35.598997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:35.598997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:35.598997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:35.598997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:35.598997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:35.598997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:35.598997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:35.598997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:35.598997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:35.601414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:35.601414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:35.601414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:35.601414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:35.601414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:35.601414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:35.601414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:35.601414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:35.601414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:35.601414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:35.601414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:20:35.601414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:35.601414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 437/891] 10.222.0.58 () {56 vars in 1120 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:35.606963 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826aefd1-afc2-4ddb-a184-2c506d63273b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:35.648249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:35.718285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826aefd1-afc2-4ddb-a184-2c506d63273b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:35.755056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847bb6f-73dc-4e6f-87c8-4142a68aa059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.756118 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847bb6f-73dc-4e6f-87c8-4142a68aa059 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.756619 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847bb6f-73dc-4e6f-87c8-4142a68aa059 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.757116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847bb6f-73dc-4e6f-87c8-4142a68aa059 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.757992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847bb6f-73dc-4e6f-87c8-4142a68aa059 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:35.758505 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847bb6f-73dc-4e6f-87c8-4142a68aa059 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.760527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847bb6f-73dc-4e6f-87c8-4142a68aa059 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.769611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 455/892] 10.222.0.58 () {56 vars in 1115 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:35.778435 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770e985-fd89-4323-b4b0-c523ad531ed1 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:35.905553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:35.911313 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770e985-fd89-4323-b4b0-c523ad531ed1 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:35.949672 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec7477-8c75-472f-814b-f2d5f19ff5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:35.950574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec7477-8c75-472f-814b-f2d5f19ff5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:35.950856 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec7477-8c75-472f-814b-f2d5f19ff5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:35.951130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec7477-8c75-472f-814b-f2d5f19ff5c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:35.962970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec7477-8c75-472f-814b-f2d5f19ff5c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:35.964444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec7477-8c75-472f-814b-f2d5f19ff5c8 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:35.979860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-7aec7477-8c75-472f-814b-f2d5f19ff5c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:35.979860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:35.979860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:35.979860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:35.979860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:35.979860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:35.979860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:35.979860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:35.979860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:35.979860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:35.979860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:35.979860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:35.979860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:35.979860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:35.979860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:35.979860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:35.980870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:35.981784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:35.981784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:35.981784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:35.981784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 438/893] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:35.987125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847bb6f-73dc-4e6f-87c8-4142a68aa059 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:36.063988 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c93ce0-dfb0-41fc-b6f0-d431f720c876 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.064341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c93ce0-dfb0-41fc-b6f0-d431f720c876 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.064539 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c93ce0-dfb0-41fc-b6f0-d431f720c876 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.065883 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 439/894] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:36.070845 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80a4ea-f53d-4475-a5af-7869e21f8b42 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.071130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80a4ea-f53d-4475-a5af-7869e21f8b42 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.071408 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80a4ea-f53d-4475-a5af-7869e21f8b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.090617 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-4d80a4ea-f53d-4475-a5af-7869e21f8b42 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:36.091837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-4d80a4ea-f53d-4475-a5af-7869e21f8b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:36.096026 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847bb6f-73dc-4e6f-87c8-4142a68aa059 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:36.127581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:36.140871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74abac-d841-4868-8e21-55e90fb62c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:36.141723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74abac-d841-4868-8e21-55e90fb62c62 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.142160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74abac-d841-4868-8e21-55e90fb62c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.142587 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74abac-d841-4868-8e21-55e90fb62c62 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.143383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74abac-d841-4868-8e21-55e90fb62c62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:36.143870 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74abac-d841-4868-8e21-55e90fb62c62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:36.145412 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74abac-d841-4868-8e21-55e90fb62c62 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:36.159985 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 456/895] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:36.165328 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74abac-d841-4868-8e21-55e90fb62c62 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:36.173277 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 440/896] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:36.280581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74abac-d841-4868-8e21-55e90fb62c62 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:36.317823 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da908005-665f-40c3-bccc-18c67d004ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:36.318838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da908005-665f-40c3-bccc-18c67d004ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.319332 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da908005-665f-40c3-bccc-18c67d004ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.319849 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da908005-665f-40c3-bccc-18c67d004ea1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.330238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da908005-665f-40c3-bccc-18c67d004ea1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:36.332131 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da908005-665f-40c3-bccc-18c67d004ea1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:36.347171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-da908005-665f-40c3-bccc-18c67d004ea1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:36.347171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:36.347171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:36.347171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:36.347171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:36.347171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:36.347171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:36.347171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:36.347171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:36.347171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:36.347171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:36.347171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:36.347171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:36.347171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:36.347171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:36.347171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:36.348189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:36.349147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:36.349147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:36.349147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:36.349147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:36.349147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:36.349147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:36.349147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:36.349147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:36.349147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:36.349147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:36.355397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 457/897] 10.222.0.58 () {56 vars in 1117 bytes} [Wed Aug 24 12:20:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:36.357838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80a4ea-f53d-4475-a5af-7869e21f8b42 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:36.447670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:36.450133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da908005-665f-40c3-bccc-18c67d004ea1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:36.470972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:36.472940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80a4ea-f53d-4475-a5af-7869e21f8b42 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:36.519037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679223e-83af-4ca5-bf5e-75bd91112957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:36.520078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679223e-83af-4ca5-bf5e-75bd91112957 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.520571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679223e-83af-4ca5-bf5e-75bd91112957 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.521033 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679223e-83af-4ca5-bf5e-75bd91112957 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.522226 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679223e-83af-4ca5-bf5e-75bd91112957 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:36.522722 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679223e-83af-4ca5-bf5e-75bd91112957 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:36.525983 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679223e-83af-4ca5-bf5e-75bd91112957 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:36.551084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 441/898] 10.222.0.58 () {56 vars in 1149 bytes} [Wed Aug 24 12:20:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:36.555872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679223e-83af-4ca5-bf5e-75bd91112957 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:36.569047 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da908005-665f-40c3-bccc-18c67d004ea1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:36.607295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d15a7-03a1-4de5-a8cd-e596816363d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:36.608158 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d15a7-03a1-4de5-a8cd-e596816363d2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.608539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d15a7-03a1-4de5-a8cd-e596816363d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.608935 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1d15a7-03a1-4de5-a8cd-e596816363d2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.609630 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d15a7-03a1-4de5-a8cd-e596816363d2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:36.611164 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1d15a7-03a1-4de5-a8cd-e596816363d2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:36.621545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-9f1d15a7-03a1-4de5-a8cd-e596816363d2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:36.621545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:36.621545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:36.621545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:36.621545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:36.621545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:36.621545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:36.621545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:36.621545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:36.621545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:36.621545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:36.621545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:36.621545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:36.621545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:36.621545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:36.621545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:36.622881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:36.624072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:36.624072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:36.624072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:36.624072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:36.624072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:36.624072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:36.624072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:36.624072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:36.624072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:36.624072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:36.624072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:36.624072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:36.624072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:36.624072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:36.625702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 458/899] 10.222.0.58 () {56 vars in 1132 bytes} [Wed Aug 24 12:20:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:36.631239 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d15a7-03a1-4de5-a8cd-e596816363d2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:36.692796 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679223e-83af-4ca5-bf5e-75bd91112957 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:36.737864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ab303-188d-44ee-bd0b-eb9637337f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:36.738755 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ab303-188d-44ee-bd0b-eb9637337f44 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.739590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ab303-188d-44ee-bd0b-eb9637337f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.740043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ab303-188d-44ee-bd0b-eb9637337f44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.743763 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:36.755828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:36.769830 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1d15a7-03a1-4de5-a8cd-e596816363d2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:36.815787 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb9c75-5540-4337-b338-c871d47fa8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:36.816907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb9c75-5540-4337-b338-c871d47fa8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.817296 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb9c75-5540-4337-b338-c871d47fa8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.817909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb9c75-5540-4337-b338-c871d47fa8f9 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.818617 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb9c75-5540-4337-b338-c871d47fa8f9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:36.819013 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb9c75-5540-4337-b338-c871d47fa8f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:36.821393 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb9c75-5540-4337-b338-c871d47fa8f9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:36.831772 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 459/900] 10.222.0.58 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:36.834109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb9c75-5540-4337-b338-c871d47fa8f9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:36.844466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ab303-188d-44ee-bd0b-eb9637337f44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:36.846673 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ab303-188d-44ee-bd0b-eb9637337f44 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:36.938947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb9c75-5540-4337-b338-c871d47fa8f9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:36.974231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435d8bc-484d-493b-9168-252b8154e1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:36.974837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435d8bc-484d-493b-9168-252b8154e1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:36.975119 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435d8bc-484d-493b-9168-252b8154e1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:36.975429 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435d8bc-484d-493b-9168-252b8154e1f1 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:36.976057 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435d8bc-484d-493b-9168-252b8154e1f1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:36.976364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435d8bc-484d-493b-9168-252b8154e1f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:36.977740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8435d8bc-484d-493b-9168-252b8154e1f1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:36.983981 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 442/902] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:36 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 12:20:36.986134 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ab303-188d-44ee-bd0b-eb9637337f44 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:36.988439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 460/902] 10.222.0.58 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:36.990210 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435d8bc-484d-493b-9168-252b8154e1f1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:37.003942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:37.035895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:37.101433 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435d8bc-484d-493b-9168-252b8154e1f1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:37.104464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ab303-188d-44ee-bd0b-eb9637337f44 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:37.139354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3535b-9897-4ea6-912e-ae8e2cb8cf3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:37.140371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3535b-9897-4ea6-912e-ae8e2cb8cf3d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.140851 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3535b-9897-4ea6-912e-ae8e2cb8cf3d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.141284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3535b-9897-4ea6-912e-ae8e2cb8cf3d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.142126 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3535b-9897-4ea6-912e-ae8e2cb8cf3d None admin] RBAC: Adding query filter params (interface=public, service_id=5e9f78def15947f5af033a0923964cc5, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:37.142576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3535b-9897-4ea6-912e-ae8e2cb8cf3d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:37.142750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf38cc-b4a2-4809-ba1b-4b8a078cc2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:37.143471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf38cc-b4a2-4809-ba1b-4b8a078cc2da None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.143870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf38cc-b4a2-4809-ba1b-4b8a078cc2da None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.144229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf38cc-b4a2-4809-ba1b-4b8a078cc2da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.144768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b3535b-9897-4ea6-912e-ae8e2cb8cf3d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:37.156836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 461/903] 10.222.0.58 () {56 vars in 1251 bytes} [Wed Aug 24 12:20:36 2022] GET /identity/v3/endpoints?service_id=5e9f78def15947f5af033a0923964cc5&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:37.159105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3535b-9897-4ea6-912e-ae8e2cb8cf3d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:37.161648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf38cc-b4a2-4809-ba1b-4b8a078cc2da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:37.163206 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf38cc-b4a2-4809-ba1b-4b8a078cc2da None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:37.180297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-d0bf38cc-b4a2-4809-ba1b-4b8a078cc2da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:37.180297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:37.180297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:37.180297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:37.180297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:37.180297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:37.180297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:37.180297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:37.180297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:37.180297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:37.180297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:37.180297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:37.180297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:37.180297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:37.180297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:37.180297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:37.182173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:37.183461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:37.183461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:37.183461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:37.188159 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 443/904] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:37.191356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf38cc-b4a2-4809-ba1b-4b8a078cc2da None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:37.258022 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:37.286308 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3535b-9897-4ea6-912e-ae8e2cb8cf3d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:37.297921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:37.307412 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf38cc-b4a2-4809-ba1b-4b8a078cc2da None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:37.326237 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d31e0b-4613-4032-b9ee-4eb3a52b2f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:37.326847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d31e0b-4613-4032-b9ee-4eb3a52b2f00 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.327126 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d31e0b-4613-4032-b9ee-4eb3a52b2f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.327397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d31e0b-4613-4032-b9ee-4eb3a52b2f00 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.328022 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d31e0b-4613-4032-b9ee-4eb3a52b2f00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:37.329387 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d31e0b-4613-4032-b9ee-4eb3a52b2f00 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:37.343203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64aaa3f-c87d-4e6e-9669-969643f667f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:37.343986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64aaa3f-c87d-4e6e-9669-969643f667f8 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.344167 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64aaa3f-c87d-4e6e-9669-969643f667f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.344358 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 462/905] 10.222.0.58 () {56 vars in 1087 bytes} [Wed Aug 24 12:20:37 2022] GET /identity/v3/services => generated 1268 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:20:37.344457 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64aaa3f-c87d-4e6e-9669-969643f667f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.345058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64aaa3f-c87d-4e6e-9669-969643f667f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:37.345359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64aaa3f-c87d-4e6e-9669-969643f667f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:37.346554 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff5ab-15d3-43bc-8ff7-f396019e0cb2 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.346776 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64aaa3f-c87d-4e6e-9669-969643f667f8 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:37.347256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff5ab-15d3-43bc-8ff7-f396019e0cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.347761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff5ab-15d3-43bc-8ff7-f396019e0cb2 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.355846 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 463/906] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:37.359121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf13521-a29f-4f35-9732-d1e48443a5bd None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.359751 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf13521-a29f-4f35-9732-d1e48443a5bd None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.360529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf13521-a29f-4f35-9732-d1e48443a5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.367332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 444/907] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:37.368211 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64aaa3f-c87d-4e6e-9669-969643f667f8 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:37.384257 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6bf13521-a29f-4f35-9732-d1e48443a5bd None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:37.386770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6bf13521-a29f-4f35-9732-d1e48443a5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:37.470778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 464/908] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:37.472590 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafeec2a-433c-4782-9064-3faf5593514e None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.472967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafeec2a-433c-4782-9064-3faf5593514e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.473332 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafeec2a-433c-4782-9064-3faf5593514e None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.474870 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 465/909] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:37.481136 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c3649-e0bc-4346-9191-dc8459933978 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.481482 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c3649-e0bc-4346-9191-dc8459933978 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.481825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c3649-e0bc-4346-9191-dc8459933978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.497919 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64aaa3f-c87d-4e6e-9669-969643f667f8 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:37.500379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f87c3649-e0bc-4346-9191-dc8459933978 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:37.500842 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f87c3649-e0bc-4346-9191-dc8459933978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:37.544137 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc0831-a887-4d80-93a5-cc8bcb7bd0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:37.544805 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc0831-a887-4d80-93a5-cc8bcb7bd0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.545082 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc0831-a887-4d80-93a5-cc8bcb7bd0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.545372 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc0831-a887-4d80-93a5-cc8bcb7bd0c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.555432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc0831-a887-4d80-93a5-cc8bcb7bd0c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:37.557284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc0831-a887-4d80-93a5-cc8bcb7bd0c4 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:37.569161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-90fc0831-a887-4d80-93a5-cc8bcb7bd0c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:37.569161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:37.569161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:37.569161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:37.569161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:37.569161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:37.569161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:37.569161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:37.569161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:37.569161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:37.569161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:37.569161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:37.569161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:37.569161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:37.569161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:37.570154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:37.571047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:37.571047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:37.571047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:37.571047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:37.571047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:37.571047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:37.571047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:37.571047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:37.571047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 445/910] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:37.580682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc0831-a887-4d80-93a5-cc8bcb7bd0c4 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:37.584398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:37.591515 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 466/911] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:37.604599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:37.687536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc0831-a887-4d80-93a5-cc8bcb7bd0c4 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:37.723788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de1a34-4d96-48a9-894a-6bd5bfc24ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:37.724394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de1a34-4d96-48a9-894a-6bd5bfc24ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.724729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de1a34-4d96-48a9-894a-6bd5bfc24ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.725004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de1a34-4d96-48a9-894a-6bd5bfc24ad7 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.725694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de1a34-4d96-48a9-894a-6bd5bfc24ad7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:37.725999 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de1a34-4d96-48a9-894a-6bd5bfc24ad7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:37.727485 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de1a34-4d96-48a9-894a-6bd5bfc24ad7 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:37.739996 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 446/912] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:37.748074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c3649-e0bc-4346-9191-dc8459933978 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:37.758253 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de1a34-4d96-48a9-894a-6bd5bfc24ad7 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:37.853176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:37.868392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c3649-e0bc-4346-9191-dc8459933978 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:37.897039 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de1a34-4d96-48a9-894a-6bd5bfc24ad7 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:37.911613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f69ef7-7851-4424-9eaa-396e7936c551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:37.912372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f69ef7-7851-4424-9eaa-396e7936c551 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.912723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f69ef7-7851-4424-9eaa-396e7936c551 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.913067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f69ef7-7851-4424-9eaa-396e7936c551 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.913872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f69ef7-7851-4424-9eaa-396e7936c551 None admin] RBAC: Adding query filter params (role.id=4c5d21de669d42169b9218cb54f295e4, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=adc6a2b3f6bd45178cc7ffb181c3c016) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:37.914249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f69ef7-7851-4424-9eaa-396e7936c551 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:37.915840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f69ef7-7851-4424-9eaa-396e7936c551 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:37.922116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 467/913] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:37 2022] GET /identity/v3/role_assignments?user.id=adc6a2b3f6bd45178cc7ffb181c3c016&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=4c5d21de669d42169b9218cb54f295e4 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:37.927352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f69ef7-7851-4424-9eaa-396e7936c551 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:37.938436 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca434af-b4cd-4d30-9426-4c34eb0d6793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:37.939760 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca434af-b4cd-4d30-9426-4c34eb0d6793 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:37.940115 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca434af-b4cd-4d30-9426-4c34eb0d6793 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:37.940458 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca434af-b4cd-4d30-9426-4c34eb0d6793 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:37.941139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca434af-b4cd-4d30-9426-4c34eb0d6793 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:37.943177 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca434af-b4cd-4d30-9426-4c34eb0d6793 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-4ca434af-b4cd-4d30-9426-4c34eb0d6793 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:37.956158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:37.957229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:37.957229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:37.957229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:37.957229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:37.957229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:37.957229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:37.957229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:37.957229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:37.957229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:37.957229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:37.957229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:37.957229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:37.957229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:37.957229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:37.957229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:37.957229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:37.958062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:37.958062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:37.958062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:37.958062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:37.958062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:37.958062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:37.958062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:37.958062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:37.958062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:37.958062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:37.958062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:37.958062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:37.958062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 12:20:37.958062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:37.959904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 447/914] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:37.965663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca434af-b4cd-4d30-9426-4c34eb0d6793 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:38.023914 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:38.082792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f69ef7-7851-4424-9eaa-396e7936c551 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:38.094790 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca434af-b4cd-4d30-9426-4c34eb0d6793 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:38.126548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c223c6-d97a-4d8b-ad95-080df40ed52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:38.127159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c223c6-d97a-4d8b-ad95-080df40ed52d None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:38.127440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c223c6-d97a-4d8b-ad95-080df40ed52d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:38.127739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c223c6-d97a-4d8b-ad95-080df40ed52d None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:38.128387 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c223c6-d97a-4d8b-ad95-080df40ed52d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:38.129394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e628f-2412-49cf-9047-a8661b3dd59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:38.129757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c223c6-d97a-4d8b-ad95-080df40ed52d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:38.130000 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e628f-2412-49cf-9047-a8661b3dd59e None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:38.130277 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e628f-2412-49cf-9047-a8661b3dd59e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:38.130548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e628f-2412-49cf-9047-a8661b3dd59e None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:38.131168 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e628f-2412-49cf-9047-a8661b3dd59e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:38.131468 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e628f-2412-49cf-9047-a8661b3dd59e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:38.132837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e628f-2412-49cf-9047-a8661b3dd59e None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:38.136607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:38.140276 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 448/915] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:38.146017 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e628f-2412-49cf-9047-a8661b3dd59e None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:38.148208 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 468/916] 10.222.0.58 () {60 vars in 1141 bytes} [Wed Aug 24 12:20:37 2022] POST /identity/v3/services => generated 241 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:38.268888 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e628f-2412-49cf-9047-a8661b3dd59e None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:38.313014 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a57ead-0e56-4e4b-a285-88e27eaf0303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:38.314005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a57ead-0e56-4e4b-a285-88e27eaf0303 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:38.314524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a57ead-0e56-4e4b-a285-88e27eaf0303 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:38.314804 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a57ead-0e56-4e4b-a285-88e27eaf0303 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:38.315423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a57ead-0e56-4e4b-a285-88e27eaf0303 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:38.315754 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a57ead-0e56-4e4b-a285-88e27eaf0303 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:38.317184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a57ead-0e56-4e4b-a285-88e27eaf0303 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:38.323385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 449/917] 10.222.0.58 () {56 vars in 1114 bytes} [Wed Aug 24 12:20:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:38.329142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c223c6-d97a-4d8b-ad95-080df40ed52d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:38.438508 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c223c6-d97a-4d8b-ad95-080df40ed52d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:38.462633 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:38.475983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfac3f2-db58-4622-b8c9-0de2c37d1df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:38.476595 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfac3f2-db58-4622-b8c9-0de2c37d1df2 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:38.476875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfac3f2-db58-4622-b8c9-0de2c37d1df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:38.477147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfac3f2-db58-4622-b8c9-0de2c37d1df2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:38.477823 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfac3f2-db58-4622-b8c9-0de2c37d1df2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:38.479930 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfac3f2-db58-4622-b8c9-0de2c37d1df2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:38.502992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 469/918] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:20:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:39.424907 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62317e-d0f9-4888-aaa5-60bbee67b54e None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:39.425460 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62317e-d0f9-4888-aaa5-60bbee67b54e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:39.425460 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62317e-d0f9-4888-aaa5-60bbee67b54e None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:39.427027 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 450/919] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:39.433134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b9d02-a410-42f2-a900-91565e61b0aa None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:39.433537 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b9d02-a410-42f2-a900-91565e61b0aa None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:39.433725 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b9d02-a410-42f2-a900-91565e61b0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:39.453069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ad4b9d02-a410-42f2-a900-91565e61b0aa None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:39.454237 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ad4b9d02-a410-42f2-a900-91565e61b0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:39.532047 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 470/920] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:39.808035 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f62317e-d0f9-4888-aaa5-60bbee67b54e None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:39.885104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:39.891456 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefeeab0-2d02-484e-a863-c49ce02fcb53 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:39.891781 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefeeab0-2d02-484e-a863-c49ce02fcb53 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:39.892060 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefeeab0-2d02-484e-a863-c49ce02fcb53 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:39.894656 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 471/921] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:39.898670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616c4df-f27d-4972-a363-4e5013754b36 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:39.898952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616c4df-f27d-4972-a363-4e5013754b36 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:39.899230 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616c4df-f27d-4972-a363-4e5013754b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:39.918129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4616c4df-f27d-4972-a363-4e5013754b36 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:39.919326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4616c4df-f27d-4972-a363-4e5013754b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:39.940048 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f62317e-d0f9-4888-aaa5-60bbee67b54e None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:39.976573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50e05f-2fe0-4b6e-847a-8d80d83aa23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:39.977210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50e05f-2fe0-4b6e-847a-8d80d83aa23d None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:39.977489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50e05f-2fe0-4b6e-847a-8d80d83aa23d None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:39.977756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50e05f-2fe0-4b6e-847a-8d80d83aa23d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:39.978928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50e05f-2fe0-4b6e-847a-8d80d83aa23d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:39.980438 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e50e05f-2fe0-4b6e-847a-8d80d83aa23d None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:39.989204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-7e50e05f-2fe0-4b6e-847a-8d80d83aa23d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:39.989204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:39.989204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:39.989204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:39.989204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:39.989204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:39.989204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:39.989204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:39.989204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:39.989204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:39.989204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:39.989204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:39.989204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:39.989204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:39.989204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:39.989204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:39.990145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:39.991031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:39.991031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:39.991031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:39.991031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:39.991031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:39.991031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:39.991031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:39.991031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:39.991031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:39.991031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:39.991031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:39.991031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:39.991031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 12:20:39.991031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:39.991031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 451/922] 10.222.0.58 () {56 vars in 1132 bytes} [Wed Aug 24 12:20:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:39.999056 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50e05f-2fe0-4b6e-847a-8d80d83aa23d None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:40.007780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 472/923] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:40.111063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50e05f-2fe0-4b6e-847a-8d80d83aa23d None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:40.136150 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:40.148731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2807e4-bb61-4173-80f3-01415ad2f765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:40.149631 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2807e4-bb61-4173-80f3-01415ad2f765 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.150847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2807e4-bb61-4173-80f3-01415ad2f765 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.151515 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2807e4-bb61-4173-80f3-01415ad2f765 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.152462 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2807e4-bb61-4173-80f3-01415ad2f765 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:40.153019 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2807e4-bb61-4173-80f3-01415ad2f765 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:40.154579 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2807e4-bb61-4173-80f3-01415ad2f765 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:40.159543 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128155c0-a9cb-4aeb-b50a-a55a0e7fd6a2 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.159859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128155c0-a9cb-4aeb-b50a-a55a0e7fd6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.160134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128155c0-a9cb-4aeb-b50a-a55a0e7fd6a2 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.160721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 452/924] 10.222.0.58 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:40.165305 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 473/925] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:40.167187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128155c0-a9cb-4aeb-b50a-a55a0e7fd6a2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:40.167911 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff98503-f87f-469b-95f6-1c152337f586 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.168391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff98503-f87f-469b-95f6-1c152337f586 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.168854 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff98503-f87f-469b-95f6-1c152337f586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.189265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-2ff98503-f87f-469b-95f6-1c152337f586 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:40.191593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2ff98503-f87f-469b-95f6-1c152337f586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:40.204708 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:40.288444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128155c0-a9cb-4aeb-b50a-a55a0e7fd6a2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:40.290160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 453/926] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:40.292321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff98503-f87f-469b-95f6-1c152337f586 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:40.333376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c64684-3875-4096-a989-b9b0177a54fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:40.333990 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c64684-3875-4096-a989-b9b0177a54fe None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.334272 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c64684-3875-4096-a989-b9b0177a54fe None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.334544 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c64684-3875-4096-a989-b9b0177a54fe None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.335174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c64684-3875-4096-a989-b9b0177a54fe None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:40.335477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c64684-3875-4096-a989-b9b0177a54fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:40.337962 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c64684-3875-4096-a989-b9b0177a54fe None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:40.347701 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 474/927] 10.222.0.58 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:40.349887 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c64684-3875-4096-a989-b9b0177a54fe None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:40.406844 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff98503-f87f-469b-95f6-1c152337f586 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:40.430429 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:40.454091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb049e-05e7-4bb3-998d-900a0996251a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:40.455302 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb049e-05e7-4bb3-998d-900a0996251a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.455806 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb049e-05e7-4bb3-998d-900a0996251a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.456507 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb049e-05e7-4bb3-998d-900a0996251a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.470740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb049e-05e7-4bb3-998d-900a0996251a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:40.473636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:40.478709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-91fb049e-05e7-4bb3-998d-900a0996251a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:20:40.478709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:40.478709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:40.478709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:40.478709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:40.478709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:40.478709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:40.478709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:40.478709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:40.478709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:40.478709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:40.478709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:40.478709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:40.478709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:40.478709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:40.478709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:40.479661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:40.479661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:40.479661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:40.479661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:40.479661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:40.479661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:40.479661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:40.479661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:40.479661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:40.479661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:40.479661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:40.479661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:40.479661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:40.479661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:40.479661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:40.479661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:40.480532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:40.480532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:40.480532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:40.480532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:40.480532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:40.480532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:40.480532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:40.480532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:40.480532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:40.480532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:40.480532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:20:40.480532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:40.481179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c64684-3875-4096-a989-b9b0177a54fe None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:40.487664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 454/928] 10.222.0.58 () {56 vars in 1120 bytes} [Wed Aug 24 12:20:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:40.490095 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb049e-05e7-4bb3-998d-900a0996251a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:40.528960 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b7399-4659-411b-a8d0-e7c55ba63695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:40.530737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b7399-4659-411b-a8d0-e7c55ba63695 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.531227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b7399-4659-411b-a8d0-e7c55ba63695 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.531764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b7399-4659-411b-a8d0-e7c55ba63695 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.532640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b7399-4659-411b-a8d0-e7c55ba63695 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:40.534227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b7399-4659-411b-a8d0-e7c55ba63695 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:40.561838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 475/929] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:20:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:20:40.563088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956d192-712d-40ef-94dd-e66d8e8dabc7 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.563374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956d192-712d-40ef-94dd-e66d8e8dabc7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.563734 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956d192-712d-40ef-94dd-e66d8e8dabc7 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.565135 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 476/930] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:40.568994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956d192-712d-40ef-94dd-e66d8e8dabc7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:40.616377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb049e-05e7-4bb3-998d-900a0996251a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:40.663268 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ad81d9-c1a3-4c3b-b7ba-36d77c07f446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:40.663823 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad81d9-c1a3-4c3b-b7ba-36d77c07f446 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.664112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad81d9-c1a3-4c3b-b7ba-36d77c07f446 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.664381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ad81d9-c1a3-4c3b-b7ba-36d77c07f446 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.665051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad81d9-c1a3-4c3b-b7ba-36d77c07f446 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:40.665395 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad81d9-c1a3-4c3b-b7ba-36d77c07f446 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:40.666848 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ad81d9-c1a3-4c3b-b7ba-36d77c07f446 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:40.675562 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 455/931] 10.222.0.58 () {56 vars in 1115 bytes} [Wed Aug 24 12:20:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:40.676858 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f34360-e949-495d-b3e1-7a9c8a457a14 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.677141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f34360-e949-495d-b3e1-7a9c8a457a14 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.677412 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f34360-e949-495d-b3e1-7a9c8a457a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.697311 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-c1f34360-e949-495d-b3e1-7a9c8a457a14 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:40.699212 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c1f34360-e949-495d-b3e1-7a9c8a457a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:40.699326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956d192-712d-40ef-94dd-e66d8e8dabc7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:40.738349 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:40.741016 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd23853-ef59-4d51-b836-9e1c91897308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:40.741667 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd23853-ef59-4d51-b836-9e1c91897308 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.742731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd23853-ef59-4d51-b836-9e1c91897308 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.742731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd23853-ef59-4d51-b836-9e1c91897308 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.742904 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd23853-ef59-4d51-b836-9e1c91897308 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:40.744256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd23853-ef59-4d51-b836-9e1c91897308 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-ffd23853-ef59-4d51-b836-9e1c91897308 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:40.754165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:40.755247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:40.755247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:40.755247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:40.755247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:40.755247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:40.755247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:40.755247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:40.755247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:40.755247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:40.755247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:40.755247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:40.755247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:40.755247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:40.755247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:40.755247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:40.755247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:40.756170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:40.756170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:40.756170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:40.756170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:40.756170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:40.756170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:40.756170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:40.756170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:40.756170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:40.756170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:40.756170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:40.756170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:40.756170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:40.756170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:40.759731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 477/932] 10.222.0.58 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:40.760581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd23853-ef59-4d51-b836-9e1c91897308 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:40.780769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:40.789257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 456/933] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:40.790100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f34360-e949-495d-b3e1-7a9c8a457a14 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:40.874427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd23853-ef59-4d51-b836-9e1c91897308 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:40.912509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f34360-e949-495d-b3e1-7a9c8a457a14 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:40.923097 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d214e6a-f358-474d-a62c-abd6ea503220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:40.923991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d214e6a-f358-474d-a62c-abd6ea503220 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.924744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d214e6a-f358-474d-a62c-abd6ea503220 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.925321 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d214e6a-f358-474d-a62c-abd6ea503220 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.939592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d214e6a-f358-474d-a62c-abd6ea503220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:40.941761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d214e6a-f358-474d-a62c-abd6ea503220 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:40.950216 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765291b-92e9-4860-bacc-bec9a311c8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:40.950972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765291b-92e9-4860-bacc-bec9a311c8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:40.951397 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765291b-92e9-4860-bacc-bec9a311c8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:40.951847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765291b-92e9-4860-bacc-bec9a311c8f2 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:40.952787 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765291b-92e9-4860-bacc-bec9a311c8f2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:40.952951 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765291b-92e9-4860-bacc-bec9a311c8f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:40.954799 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765291b-92e9-4860-bacc-bec9a311c8f2 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:40.961800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-1d214e6a-f358-474d-a62c-abd6ea503220 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:40.961800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:40.961800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:40.961800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:40.961800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:40.961800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:40.961800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:40.961800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:40.961800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:40.961800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:40.961800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:40.961800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:40.961800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:40.961800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:40.961800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:40.961800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:40.962838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:40.963778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:40.963778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:40.963778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:40.964400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 457/934] 10.222.0.58 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:40.965842 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 478/935] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:40.966104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765291b-92e9-4860-bacc-bec9a311c8f2 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:40.971056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d214e6a-f358-474d-a62c-abd6ea503220 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:41.002106 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:41.051214 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:41.084663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d214e6a-f358-474d-a62c-abd6ea503220 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:41.086247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765291b-92e9-4860-bacc-bec9a311c8f2 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:41.122560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d8bf7-5eac-45b4-9c51-9a6a7b965819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:41.122847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbb76e-ab18-4152-959c-5ca895bbba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:41.123270 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d8bf7-5eac-45b4-9c51-9a6a7b965819 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.123465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbb76e-ab18-4152-959c-5ca895bbba69 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.123769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d8bf7-5eac-45b4-9c51-9a6a7b965819 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.123861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbb76e-ab18-4152-959c-5ca895bbba69 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.123943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263d8bf7-5eac-45b4-9c51-9a6a7b965819 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.124115 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbb76e-ab18-4152-959c-5ca895bbba69 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.124472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d8bf7-5eac-45b4-9c51-9a6a7b965819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:41.124708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbb76e-ab18-4152-959c-5ca895bbba69 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:41.124811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d8bf7-5eac-45b4-9c51-9a6a7b965819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:41.125018 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbb76e-ab18-4152-959c-5ca895bbba69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:41.126120 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263d8bf7-5eac-45b4-9c51-9a6a7b965819 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:41.126362 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbb76e-ab18-4152-959c-5ca895bbba69 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:41.135891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 458/936] 10.222.0.58 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:41.135891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbb76e-ab18-4152-959c-5ca895bbba69 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:41.142493 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 479/937] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:41.147099 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d8bf7-5eac-45b4-9c51-9a6a7b965819 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:41.237896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:41.245513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbb76e-ab18-4152-959c-5ca895bbba69 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:41.258039 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263d8bf7-5eac-45b4-9c51-9a6a7b965819 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:41.281903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2ee34-6326-4e67-b71b-fd497bc36cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:41.282509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2ee34-6326-4e67-b71b-fd497bc36cc9 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.282795 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2ee34-6326-4e67-b71b-fd497bc36cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.283072 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2ee34-6326-4e67-b71b-fd497bc36cc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.283733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2ee34-6326-4e67-b71b-fd497bc36cc9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:41.285156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2ee34-6326-4e67-b71b-fd497bc36cc9 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:41.287765 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: WARNING keystone.server.flask.common [None req-10d2ee34-6326-4e67-b71b-fd497bc36cc9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 12:20:41.290041 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:41.297900 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0051cc3-a93d-4cde-9ce5-184885647d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:41.298741 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0051cc3-a93d-4cde-9ce5-184885647d29 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.299169 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0051cc3-a93d-4cde-9ce5-184885647d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.299587 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0051cc3-a93d-4cde-9ce5-184885647d29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.300456 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0051cc3-a93d-4cde-9ce5-184885647d29 None admin] RBAC: Adding query filter params (interface=public, service_id=cab6accadd1d4d9fa5a1830be2fa5203, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:41.301033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0051cc3-a93d-4cde-9ce5-184885647d29 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:41.302705 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0051cc3-a93d-4cde-9ce5-184885647d29 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:41.304898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.sql.core [None req-10d2ee34-6326-4e67-b71b-fd497bc36cc9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 12:20:41.304898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 12:20:41.304898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [parameters: {'id': 'e96cd639891248dfa255b40af538c86d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 12:20:41.304898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81637) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 12:20:41.309476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 480/938] 10.222.0.58 () {56 vars in 1251 bytes} [Wed Aug 24 12:20:41 2022] GET /identity/v3/endpoints?service_id=cab6accadd1d4d9fa5a1830be2fa5203&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:41.310541 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0051cc3-a93d-4cde-9ce5-184885647d29 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:41.312885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-10d2ee34-6326-4e67-b71b-fd497bc36cc9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e96cd639891248dfa255b40af538c86d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e96cd639891248dfa255b40af538c86d. Aug 24 12:20:41.312885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:41.312885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:20:41.312885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:20:41.312885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:20:41.312885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:20:41.312885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:20:41.312885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:20:41.312885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:20:41.312885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:20:41.312885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:20:41.312885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:20:41.312885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application result.read() Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 12:20:41.313697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application yield resource Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise self.value Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application session.commit() Aug 24 12:20:41.319547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self.session.flush() Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application compat.raise_( Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 12:20:41.320584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 12:20:41.321749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 12:20:41.321749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 12:20:41.321749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 12:20:41.321749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 12:20:41.321749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 12:20:41.321749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 12:20:41.321749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 12:20:41.321749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 12:20:41.321749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 12:20:41.321749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 12:20:41.321749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 12:20:41.321749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 12:20:41.321749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 12:20:41.321749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 12:20:41.321749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application result.read() Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:20:41.323005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:20:41.324146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:20:41.324146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:20:41.324146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:20:41.324146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:20:41.324146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:20:41.324146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:20:41.324146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 12:20:41.324146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 12:20:41.324146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [parameters: {'id': 'e96cd639891248dfa255b40af538c86d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 12:20:41.324146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 12:20:41.324146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:41.324146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:41.324146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:41.324146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:41.324146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:41.325509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:41.326648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:41.326648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 12:20:41.326648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 12:20:41.326648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:41.326648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:41.326648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 12:20:41.326648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 12:20:41.326648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e96cd639891248dfa255b40af538c86d. Aug 24 12:20:41.326648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:41.326648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 459/939] 10.222.0.58 () {60 vars in 1140 bytes} [Wed Aug 24 12:20:41 2022] POST /identity/v3/projects => generated 262 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:41.326648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2ee34-6326-4e67-b71b-fd497bc36cc9 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:41.423550 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2ee34-6326-4e67-b71b-fd497bc36cc9 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:41.433761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0051cc3-a93d-4cde-9ce5-184885647d29 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:41.462024 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748b239-c537-4476-b82c-975e365804ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:41.462738 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748b239-c537-4476-b82c-975e365804ab None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.462927 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748b239-c537-4476-b82c-975e365804ab None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.463204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748b239-c537-4476-b82c-975e365804ab None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.463849 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748b239-c537-4476-b82c-975e365804ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:41.465412 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748b239-c537-4476-b82c-975e365804ab None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:41.469173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf52ac2-2739-4373-8873-5062bee56234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:41.469798 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf52ac2-2739-4373-8873-5062bee56234 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.470074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf52ac2-2739-4373-8873-5062bee56234 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.470340 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf52ac2-2739-4373-8873-5062bee56234 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.474399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 460/940] 10.222.0.58 () {56 vars in 1087 bytes} [Wed Aug 24 12:20:41 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:20:41.475109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748b239-c537-4476-b82c-975e365804ab None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:41.489795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf52ac2-2739-4373-8873-5062bee56234 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:41.492655 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf52ac2-2739-4373-8873-5062bee56234 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:41.494647 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:41.513491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-7cf52ac2-2739-4373-8873-5062bee56234 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:41.513491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:41.513491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:41.513491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:41.513491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:41.513491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:41.513491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:41.513491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:41.513491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:41.513491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:41.513491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:41.513491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:41.513491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:41.513491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:41.513491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:41.513491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:41.514718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:41.515915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:41.515915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:41.515915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:41.515915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:41.515915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:41.515915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:41.515915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:41.515915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:41.515915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:41.515915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:41.523723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 481/941] 10.222.0.58 () {56 vars in 1117 bytes} [Wed Aug 24 12:20:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:41.526570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf52ac2-2739-4373-8873-5062bee56234 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:41.608724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:41.612153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748b239-c537-4476-b82c-975e365804ab None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:41.655802 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc2ee6-f5b1-40a2-a1a4-6fa48053902d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:41.656893 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc2ee6-f5b1-40a2-a1a4-6fa48053902d None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.657187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc2ee6-f5b1-40a2-a1a4-6fa48053902d None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.657445 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc2ee6-f5b1-40a2-a1a4-6fa48053902d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.665176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc2ee6-f5b1-40a2-a1a4-6fa48053902d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:41.666726 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc2ee6-f5b1-40a2-a1a4-6fa48053902d None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:41.670021 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf52ac2-2739-4373-8873-5062bee56234 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:41.678201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-03bc2ee6-f5b1-40a2-a1a4-6fa48053902d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:41.678201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:41.678201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:41.678201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:41.678201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:41.678201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:41.678201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:41.678201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:41.678201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:41.678201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:41.678201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:41.678201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:41.678201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:41.678201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:41.678201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:41.679118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:41.680029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:41.680029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:41.680029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:41.680029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:41.680029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:41.680029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:41.680029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:41.680029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:41.680029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 461/942] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:41.686136 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc2ee6-f5b1-40a2-a1a4-6fa48053902d None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:41.716770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a4c13-d8e2-4326-a058-b78112c7f591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:41.717507 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a4c13-d8e2-4326-a058-b78112c7f591 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.718069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a4c13-d8e2-4326-a058-b78112c7f591 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.718346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a4c13-d8e2-4326-a058-b78112c7f591 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.719653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a4c13-d8e2-4326-a058-b78112c7f591 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:41.720297 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a4c13-d8e2-4326-a058-b78112c7f591 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:41.722533 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a4c13-d8e2-4326-a058-b78112c7f591 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:41.747835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 482/943] 10.222.0.58 () {56 vars in 1149 bytes} [Wed Aug 24 12:20:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:41.753358 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a4c13-d8e2-4326-a058-b78112c7f591 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:41.770293 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:41.796983 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc2ee6-f5b1-40a2-a1a4-6fa48053902d None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:41.835913 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e8141-fcd9-4160-8ec8-4501bf128583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:41.836744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e8141-fcd9-4160-8ec8-4501bf128583 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.837107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e8141-fcd9-4160-8ec8-4501bf128583 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.837624 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e8141-fcd9-4160-8ec8-4501bf128583 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.838441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e8141-fcd9-4160-8ec8-4501bf128583 None admin] RBAC: Adding query filter params (name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:41.838921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e8141-fcd9-4160-8ec8-4501bf128583 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:41.840605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e8141-fcd9-4160-8ec8-4501bf128583 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:41.864779 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 462/944] 10.222.0.58 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:41.891666 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a4c13-d8e2-4326-a058-b78112c7f591 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:41.903979 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:41.927444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a986f-313b-427f-8380-ab229a6652ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:41.928163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a986f-313b-427f-8380-ab229a6652ab None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:41.928501 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a986f-313b-427f-8380-ab229a6652ab None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:41.928786 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a986f-313b-427f-8380-ab229a6652ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:41.941612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a986f-313b-427f-8380-ab229a6652ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:41.943132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a986f-313b-427f-8380-ab229a6652ab None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:41.961361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d61a986f-313b-427f-8380-ab229a6652ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:41.961361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:41.961361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:41.961361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:41.961361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:41.961361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:41.961361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:41.961361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:41.961361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:41.961361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:41.961361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:41.961361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:41.961361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:41.961361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:41.961361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:41.961361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:41.962415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:41.963330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:41.963330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:41.963330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:41.963330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 483/945] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:41.968615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e8141-fcd9-4160-8ec8-4501bf128583 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:42.076797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e8141-fcd9-4160-8ec8-4501bf128583 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:42.112559 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66ddaa-509f-4bfc-bc31-83bd29d08772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:42.113167 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66ddaa-509f-4bfc-bc31-83bd29d08772 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:42.113447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66ddaa-509f-4bfc-bc31-83bd29d08772 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:42.113719 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b66ddaa-509f-4bfc-bc31-83bd29d08772 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:42.114360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66ddaa-509f-4bfc-bc31-83bd29d08772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:42.114659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66ddaa-509f-4bfc-bc31-83bd29d08772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:42.116135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b66ddaa-509f-4bfc-bc31-83bd29d08772 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:42.123988 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:42.129386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 463/946] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:42.135168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a986f-313b-427f-8380-ab229a6652ab None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:42.244768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a986f-313b-427f-8380-ab229a6652ab None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:42.280236 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361a3d6-515b-488c-b001-5e90d7b638e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:42.281115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361a3d6-515b-488c-b001-5e90d7b638e3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:42.281569 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361a3d6-515b-488c-b001-5e90d7b638e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:42.281989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361a3d6-515b-488c-b001-5e90d7b638e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:42.289095 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361a3d6-515b-488c-b001-5e90d7b638e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:42.291299 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361a3d6-515b-488c-b001-5e90d7b638e3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:42.303105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e361a3d6-515b-488c-b001-5e90d7b638e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:42.303105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:42.303105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:42.303105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:42.303105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:42.303105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:42.303105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:42.303105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:42.303105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:42.303105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:42.303105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:42.303105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:42.303105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:42.303105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:42.303105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:42.304258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:42.305389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:42.305389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:42.305389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:42.305389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:42.305389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:42.305389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:42.305389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:42.305389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:42.308052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 484/947] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:42.316822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66ddaa-509f-4bfc-bc31-83bd29d08772 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:42.421456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b66ddaa-509f-4bfc-bc31-83bd29d08772 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:42.456177 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5028e-1a1d-49bd-bd56-47bbf9c476dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:42.456781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5028e-1a1d-49bd-bd56-47bbf9c476dd None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:42.457064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5028e-1a1d-49bd-bd56-47bbf9c476dd None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:42.457394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5028e-1a1d-49bd-bd56-47bbf9c476dd None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:42.458088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5028e-1a1d-49bd-bd56-47bbf9c476dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:42.458496 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5028e-1a1d-49bd-bd56-47bbf9c476dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:42.460038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5028e-1a1d-49bd-bd56-47bbf9c476dd None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:42.473581 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 464/948] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:42.479883 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361a3d6-515b-488c-b001-5e90d7b638e3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:42.505492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:42.591268 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361a3d6-515b-488c-b001-5e90d7b638e3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:42.607006 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550f2d8-eb26-4c96-8bf2-20a7232e1b8c None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:42.607006 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550f2d8-eb26-4c96-8bf2-20a7232e1b8c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:42.607290 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550f2d8-eb26-4c96-8bf2-20a7232e1b8c None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:42.608870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 465/949] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:42.614212 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd57a95-83f9-44e0-9b2b-3010d254adc3 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:42.614353 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd57a95-83f9-44e0-9b2b-3010d254adc3 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:42.614537 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd57a95-83f9-44e0-9b2b-3010d254adc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:42.630184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cefabc-8c7a-4e34-9f8c-2f896331738a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:42.631499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cefabc-8c7a-4e34-9f8c-2f896331738a None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:42.631985 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cefabc-8c7a-4e34-9f8c-2f896331738a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:42.632473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cefabc-8c7a-4e34-9f8c-2f896331738a None admin] PATH_INFO: `/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/adc6a2b3f6bd45178cc7ffb181c3c016/roles/4c5d21de669d42169b9218cb54f295e4` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:42.634874 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cefabc-8c7a-4e34-9f8c-2f896331738a None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user_id=adc6a2b3f6bd45178cc7ffb181c3c016, role_id=4c5d21de669d42169b9218cb54f295e4)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:42.635309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-acd57a95-83f9-44e0-9b2b-3010d254adc3 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:42.636757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cefabc-8c7a-4e34-9f8c-2f896331738a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:42.636867 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-acd57a95-83f9-44e0-9b2b-3010d254adc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:42.656023 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 485/950] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:20:42 2022] PUT /identity/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/adc6a2b3f6bd45178cc7ffb181c3c016/roles/4c5d21de669d42169b9218cb54f295e4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:20:42.701992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:42.726948 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 466/951] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:43.022722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cefabc-8c7a-4e34-9f8c-2f896331738a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:43.117303 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:43.136350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cefabc-8c7a-4e34-9f8c-2f896331738a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:43.173504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb50bc1-9de7-4fcf-ad4f-36860ed9ca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:43.174179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb50bc1-9de7-4fcf-ad4f-36860ed9ca02 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.174457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb50bc1-9de7-4fcf-ad4f-36860ed9ca02 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.174724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb50bc1-9de7-4fcf-ad4f-36860ed9ca02 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:43.175346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb50bc1-9de7-4fcf-ad4f-36860ed9ca02 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:43.176826 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb50bc1-9de7-4fcf-ad4f-36860ed9ca02 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-adb50bc1-9de7-4fcf-ad4f-36860ed9ca02 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:43.186254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:43.187300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:43.187300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:43.187300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:43.187300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:43.187300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:43.187300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:43.187300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:43.187300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:43.187300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:43.187300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:43.187300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:43.187300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:43.187300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:43.187300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:43.187300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:43.187300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:43.188582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:43.188582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:43.188582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:43.188582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:43.188582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:43.188582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:43.188582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:43.188582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:43.188582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:43.188582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:43.188582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:43.188582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:43.188582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:43.188582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:43.188582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 486/952] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:43.193760 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd57a95-83f9-44e0-9b2b-3010d254adc3 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:43.301410 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd57a95-83f9-44e0-9b2b-3010d254adc3 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:43.340392 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c863c-ef59-412e-b5b7-9e233da1a564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:43.341061 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c863c-ef59-412e-b5b7-9e233da1a564 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.341355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c863c-ef59-412e-b5b7-9e233da1a564 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.341627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c863c-ef59-412e-b5b7-9e233da1a564 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:43.342261 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c863c-ef59-412e-b5b7-9e233da1a564 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:43.342560 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c863c-ef59-412e-b5b7-9e233da1a564 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:43.344008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c863c-ef59-412e-b5b7-9e233da1a564 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:43.349332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 467/953] 10.222.0.58 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:43.356050 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb50bc1-9de7-4fcf-ad4f-36860ed9ca02 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:43.460753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb50bc1-9de7-4fcf-ad4f-36860ed9ca02 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:43.497153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172dcb0-fa3a-47e8-9df1-11ce66c45ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:43.497758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172dcb0-fa3a-47e8-9df1-11ce66c45ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.498035 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172dcb0-fa3a-47e8-9df1-11ce66c45ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.498301 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172dcb0-fa3a-47e8-9df1-11ce66c45ac8 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:43.498929 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172dcb0-fa3a-47e8-9df1-11ce66c45ac8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:43.499234 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172dcb0-fa3a-47e8-9df1-11ce66c45ac8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:43.500628 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172dcb0-fa3a-47e8-9df1-11ce66c45ac8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:43.505504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 487/954] 10.222.0.58 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:43.579071 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f9ed3-b4fe-458f-a6af-d1a3bcc7dba7 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.580103 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f9ed3-b4fe-458f-a6af-d1a3bcc7dba7 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.580591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f9ed3-b4fe-458f-a6af-d1a3bcc7dba7 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:43.582556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 468/955] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:43.589373 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09317a-fea0-49bf-9105-13b99bdbbedb None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.590349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09317a-fea0-49bf-9105-13b99bdbbedb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.590855 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09317a-fea0-49bf-9105-13b99bdbbedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:43.610561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7d09317a-fea0-49bf-9105-13b99bdbbedb None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:43.612160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7d09317a-fea0-49bf-9105-13b99bdbbedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:43.621097 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:43.693612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 488/956] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:43.933026 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6cc5b-8a33-44e3-8dc9-f435e8a8bd0e None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.933378 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6cc5b-8a33-44e3-8dc9-f435e8a8bd0e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.933585 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b6cc5b-8a33-44e3-8dc9-f435e8a8bd0e None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:43.935070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 469/957] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:43.940507 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dadeae-6937-45aa-88d1-ea16dbe724e5 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:43.941039 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dadeae-6937-45aa-88d1-ea16dbe724e5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:43.941519 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dadeae-6937-45aa-88d1-ea16dbe724e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:43.960270 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a6dadeae-6937-45aa-88d1-ea16dbe724e5 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:43.961723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a6dadeae-6937-45aa-88d1-ea16dbe724e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:43.974175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6cc5b-8a33-44e3-8dc9-f435e8a8bd0e None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:44.030807 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:44.058964 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 489/958] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:44.091792 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b6cc5b-8a33-44e3-8dc9-f435e8a8bd0e None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:44.127336 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157d034-832a-4e08-9998-84aa01217141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:44.128052 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157d034-832a-4e08-9998-84aa01217141 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.128336 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157d034-832a-4e08-9998-84aa01217141 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.128609 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4157d034-832a-4e08-9998-84aa01217141 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.129186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157d034-832a-4e08-9998-84aa01217141 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:44.130603 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4157d034-832a-4e08-9998-84aa01217141 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-4157d034-832a-4e08-9998-84aa01217141 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:44.140043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:44.141131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:44.141131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:44.141131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:44.141131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:44.141131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:44.141131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:44.141131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:44.141131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:44.141131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:44.141131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:44.141131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:44.141131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:44.141131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:44.141131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:44.141131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:44.141131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:44.142036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:44.142036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:44.142036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:44.142036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:44.142036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:44.142036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:44.142036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:44.142036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:44.142036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:44.142036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:44.142036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:44.142036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:44.142036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:44.142036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:44.142036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 470/959] 10.222.0.58 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:44.147379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dadeae-6937-45aa-88d1-ea16dbe724e5 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:44.227447 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:44.258834 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dadeae-6937-45aa-88d1-ea16dbe724e5 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:44.295439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e0f04-3684-443a-94f1-8f30c3a32ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:44.296412 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e0f04-3684-443a-94f1-8f30c3a32ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.296888 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e0f04-3684-443a-94f1-8f30c3a32ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.297456 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e0f04-3684-443a-94f1-8f30c3a32ab8 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.298270 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e0f04-3684-443a-94f1-8f30c3a32ab8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:44.298767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e0f04-3684-443a-94f1-8f30c3a32ab8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:44.300348 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494e0f04-3684-443a-94f1-8f30c3a32ab8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:44.305621 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 490/960] 10.222.0.58 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:44.313918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157d034-832a-4e08-9998-84aa01217141 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:44.390603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e0f04-3684-443a-94f1-8f30c3a32ab8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:44.425338 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4157d034-832a-4e08-9998-84aa01217141 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:44.441329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:44.466368 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08fbea-c11d-4985-b35d-3c00c6d8a0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:44.467300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08fbea-c11d-4985-b35d-3c00c6d8a0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.467843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08fbea-c11d-4985-b35d-3c00c6d8a0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.468316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed08fbea-c11d-4985-b35d-3c00c6d8a0c2 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.469199 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08fbea-c11d-4985-b35d-3c00c6d8a0c2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:44.469732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08fbea-c11d-4985-b35d-3c00c6d8a0c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:44.471292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed08fbea-c11d-4985-b35d-3c00c6d8a0c2 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:44.477009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 471/961] 10.222.0.58 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:44.482863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08fbea-c11d-4985-b35d-3c00c6d8a0c2 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:44.523679 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e0f04-3684-443a-94f1-8f30c3a32ab8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:44.567904 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9c99a-f7d4-4c5b-860d-9cbe8a074ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:44.569000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9c99a-f7d4-4c5b-860d-9cbe8a074ffb None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.569407 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9c99a-f7d4-4c5b-860d-9cbe8a074ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.569680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9c99a-f7d4-4c5b-860d-9cbe8a074ffb None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.570332 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e9c99a-f7d4-4c5b-860d-9cbe8a074ffb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:44.573280 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e9c99a-f7d4-4c5b-860d-9cbe8a074ffb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:44.577384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: WARNING keystone.server.flask.common [None req-d4e9c99a-f7d4-4c5b-860d-9cbe8a074ffb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 12:20:44.589728 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:44.594446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.sql.core [None req-d4e9c99a-f7d4-4c5b-860d-9cbe8a074ffb None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 12:20:44.594446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 12:20:44.594446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [parameters: {'user_id': 'aa0c2163577d4a388cf8958396ab95fb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 12:20:44.594446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81638) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 12:20:44.599166 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed08fbea-c11d-4985-b35d-3c00c6d8a0c2 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:44.620838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d4e9c99a-f7d4-4c5b-860d-9cbe8a074ffb None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 12:20:44.620838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:44.620838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:20:44.620838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:20:44.620838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:20:44.620838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:20:44.620838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:20:44.620838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:20:44.620838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:20:44.620838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:20:44.620838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:20:44.620838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:20:44.620838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:20:44.620838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result.read() Aug 24 12:20:44.620838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:20:44.621824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application yield resource Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application next(self.gen) Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise self.value Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application session.commit() Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 12:20:44.622821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.session.flush() Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application compat.raise_( Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 12:20:44.623754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 12:20:44.624663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 12:20:44.624663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 12:20:44.624663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 12:20:44.624663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 12:20:44.624663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 12:20:44.624663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 12:20:44.624663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 12:20:44.624663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 12:20:44.624663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 12:20:44.624663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 12:20:44.624663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 12:20:44.624663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 12:20:44.624663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 12:20:44.624663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 12:20:44.624663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 12:20:44.624663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception Aug 24 12:20:44.625548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 12:20:44.625548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 12:20:44.625548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 12:20:44.625548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 12:20:44.625548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 12:20:44.625548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 12:20:44.625548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 12:20:44.625548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application conn.query(q) Aug 24 12:20:44.625548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 12:20:44.625548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 12:20:44.625548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 12:20:44.625548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application result.read() Aug 24 12:20:44.625548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 12:20:44.625548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 12:20:44.625548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 12:20:44.625548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 12:20:44.626396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 12:20:44.626396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 12:20:44.626396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 12:20:44.626396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 12:20:44.626396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 12:20:44.626396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 12:20:44.626396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [parameters: {'user_id': 'aa0c2163577d4a388cf8958396ab95fb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 12:20:44.626396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 12:20:44.626396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:44.626396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:44.626396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:44.626396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:44.626396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:44.626396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:44.626396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 12:20:44.627190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 12:20:44.630028 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 12:20:44.630028 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 12:20:44.630028 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 12:20:44.630028 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 12:20:44.630028 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 12:20:44.630028 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:44.632517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 491/962] 10.222.0.58 () {60 vars in 1131 bytes} [Wed Aug 24 12:20:44 2022] POST /identity/v3/users => generated 160 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:44.639807 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9c99a-f7d4-4c5b-860d-9cbe8a074ffb None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:44.647954 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85d49b-fb5f-4172-91fe-0e171dada3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:44.648504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85d49b-fb5f-4172-91fe-0e171dada3cd None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.648781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85d49b-fb5f-4172-91fe-0e171dada3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.649052 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85d49b-fb5f-4172-91fe-0e171dada3cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.650432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85d49b-fb5f-4172-91fe-0e171dada3cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:44.652282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85d49b-fb5f-4172-91fe-0e171dada3cd None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:44.679877 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 472/963] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:20:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:44.679877 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a9a08-618f-4662-a1e0-73af3f7b425d None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.679877 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a9a08-618f-4662-a1e0-73af3f7b425d None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.679877 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a9a08-618f-4662-a1e0-73af3f7b425d None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.679877 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 473/964] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:44.686345 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c647d8-ce13-4fe1-8a5c-2a654d7695e6 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.686633 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c647d8-ce13-4fe1-8a5c-2a654d7695e6 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.686915 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c647d8-ce13-4fe1-8a5c-2a654d7695e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.708758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-c5c647d8-ce13-4fe1-8a5c-2a654d7695e6 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:44.710166 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c5c647d8-ce13-4fe1-8a5c-2a654d7695e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:44.747612 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:44.771402 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9c99a-f7d4-4c5b-860d-9cbe8a074ffb None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:44.812276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5bd0d-9613-4841-83bf-0279690ff098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:44.813358 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5bd0d-9613-4841-83bf-0279690ff098 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.813848 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5bd0d-9613-4841-83bf-0279690ff098 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.814340 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f5bd0d-9613-4841-83bf-0279690ff098 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.817419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 474/965] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:44.823968 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5bd0d-9613-4841-83bf-0279690ff098 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:44.826292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f5bd0d-9613-4841-83bf-0279690ff098 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:44.839602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-57f5bd0d-9613-4841-83bf-0279690ff098 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:20:44.839602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:44.839602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:44.839602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:44.839602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:44.839602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:44.839602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:44.839602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:44.839602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:44.839602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:44.839602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:44.839602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:44.839602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:44.839602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:44.839602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:44.839602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:44.840636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:44.841599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:44.841599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:44.841599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:44.841599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:44.841599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:44.841599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:44.841599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:44.841599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:44.841599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 12:20:44.841599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:44.843514 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 492/966] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:44.849538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c647d8-ce13-4fe1-8a5c-2a654d7695e6 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:44.956559 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c647d8-ce13-4fe1-8a5c-2a654d7695e6 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:44.996196 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a9589-7feb-4cbd-acc1-3ad5888a4a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:44.996804 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a9589-7feb-4cbd-acc1-3ad5888a4a3f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:44.997081 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a9589-7feb-4cbd-acc1-3ad5888a4a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:44.997352 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a9589-7feb-4cbd-acc1-3ad5888a4a3f None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:44.998053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a9589-7feb-4cbd-acc1-3ad5888a4a3f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:44.998354 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a9589-7feb-4cbd-acc1-3ad5888a4a3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:44.999996 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a9589-7feb-4cbd-acc1-3ad5888a4a3f None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:45.035970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 475/967] 10.222.0.58 () {56 vars in 1101 bytes} [Wed Aug 24 12:20:44 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:45.114821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5bd0d-9613-4841-83bf-0279690ff098 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:45.186705 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:45.235939 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f5bd0d-9613-4841-83bf-0279690ff098 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:45.280276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680a36e-ffa3-4088-a04c-3847d318f4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:45.280924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680a36e-ffa3-4088-a04c-3847d318f4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:45.281203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680a36e-ffa3-4088-a04c-3847d318f4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:45.281551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680a36e-ffa3-4088-a04c-3847d318f4b0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:45.293383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680a36e-ffa3-4088-a04c-3847d318f4b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:45.298735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a680a36e-ffa3-4088-a04c-3847d318f4b0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:20:45.298735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:45.298735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:45.298735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:45.298735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:45.298735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:45.298735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:45.298735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:45.298735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:45.298735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:45.298735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:45.298735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:45.298735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:45.298735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:45.298735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:45.298735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:45.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:45.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:45.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:45.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:45.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:45.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:45.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:45.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:45.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:45.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:45.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:45.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:45.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:45.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:45.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:45.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:45.301174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:45.301174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:45.301174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:45.301174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:45.301174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:45.301174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:45.301174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:45.301174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:45.301174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:45.301174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:45.301174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:20:45.301174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:45.301174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 493/968] 10.222.0.58 () {56 vars in 1120 bytes} [Wed Aug 24 12:20:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:45.305985 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a9589-7feb-4cbd-acc1-3ad5888a4a3f None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:45.326046 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:45.410718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a9589-7feb-4cbd-acc1-3ad5888a4a3f None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:45.444993 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab4fa4-c04c-41c5-87bb-de4021c10b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:45.445603 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab4fa4-c04c-41c5-87bb-de4021c10b97 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:45.445889 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab4fa4-c04c-41c5-87bb-de4021c10b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:45.446164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab4fa4-c04c-41c5-87bb-de4021c10b97 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:45.447399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab4fa4-c04c-41c5-87bb-de4021c10b97 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:45.447734 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab4fa4-c04c-41c5-87bb-de4021c10b97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:45.449117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab4fa4-c04c-41c5-87bb-de4021c10b97 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:45.462176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 476/969] 10.222.0.58 () {56 vars in 1115 bytes} [Wed Aug 24 12:20:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:45.463807 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680a36e-ffa3-4088-a04c-3847d318f4b0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:45.507828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313190b-a9dc-4dc0-91e1-83030399e895 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:45.508114 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313190b-a9dc-4dc0-91e1-83030399e895 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:45.508302 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313190b-a9dc-4dc0-91e1-83030399e895 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:45.510563 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 477/970] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:45.515287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1340c58-2238-4946-950e-a8bf3bce6b56 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:45.515575 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1340c58-2238-4946-950e-a8bf3bce6b56 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:45.515886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1340c58-2238-4946-950e-a8bf3bce6b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:45.534193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-e1340c58-2238-4946-950e-a8bf3bce6b56 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:45.535394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e1340c58-2238-4946-950e-a8bf3bce6b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:45.575345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680a36e-ffa3-4088-a04c-3847d318f4b0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:45.592690 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:45.611663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 478/971] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:45.615225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd381264-df84-48d5-bd98-4976d90e3dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:45.615886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd381264-df84-48d5-bd98-4976d90e3dbd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:45.616203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd381264-df84-48d5-bd98-4976d90e3dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:45.616538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd381264-df84-48d5-bd98-4976d90e3dbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:45.627955 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd381264-df84-48d5-bd98-4976d90e3dbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:45.629399 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd381264-df84-48d5-bd98-4976d90e3dbd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:45.644043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-bd381264-df84-48d5-bd98-4976d90e3dbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:45.644043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:45.644043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:45.644043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:45.644043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:45.644043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:45.644043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:45.644043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:45.644043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:45.644043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:45.644043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:45.644043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:45.644043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:45.644043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:45.644043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:45.644043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:45.644998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:45.650835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:45.650835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:45.650835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:45.650835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 494/972] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:45.651682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1340c58-2238-4946-950e-a8bf3bce6b56 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:45.662698 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:45.755797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1340c58-2238-4946-950e-a8bf3bce6b56 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:45.791544 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f036ea-fec3-4a8c-9847-5b61bfe5b5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:45.792108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f036ea-fec3-4a8c-9847-5b61bfe5b5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:45.792394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f036ea-fec3-4a8c-9847-5b61bfe5b5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:45.792671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f036ea-fec3-4a8c-9847-5b61bfe5b5a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:45.793316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f036ea-fec3-4a8c-9847-5b61bfe5b5a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:45.793621 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f036ea-fec3-4a8c-9847-5b61bfe5b5a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:45.794991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f036ea-fec3-4a8c-9847-5b61bfe5b5a6 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:45.807235 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 479/973] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:45.812650 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd381264-df84-48d5-bd98-4976d90e3dbd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:45.883982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f036ea-fec3-4a8c-9847-5b61bfe5b5a6 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:45.924555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd381264-df84-48d5-bd98-4976d90e3dbd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:45.962482 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:45.962719 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b3c51-5193-4279-84ba-1a9ec4307f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:45.963145 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b3c51-5193-4279-84ba-1a9ec4307f06 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:45.963705 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b3c51-5193-4279-84ba-1a9ec4307f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:45.963817 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3b3c51-5193-4279-84ba-1a9ec4307f06 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:45.971652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b3c51-5193-4279-84ba-1a9ec4307f06 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:45.973103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3b3c51-5193-4279-84ba-1a9ec4307f06 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:45.984804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-4d3b3c51-5193-4279-84ba-1a9ec4307f06 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:45.984804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:45.984804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:45.984804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:45.984804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:45.984804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:45.984804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:45.984804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:45.984804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:45.984804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:45.984804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:45.984804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:45.984804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:45.984804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:45.984804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:45.984804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:45.985770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:45.986753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:45.986753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:45.986753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:45.986753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:45.986753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:45.986753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:45.986753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:45.986753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:45.986753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 12:20:45.986753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:45.986753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 495/974] 10.222.0.58 () {56 vars in 1117 bytes} [Wed Aug 24 12:20:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:45.991454 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b3c51-5193-4279-84ba-1a9ec4307f06 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:46.004259 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f036ea-fec3-4a8c-9847-5b61bfe5b5a6 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:46.028463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:46.044531 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0050a7-4fc3-447c-a55d-ada3db523659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:46.045077 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0050a7-4fc3-447c-a55d-ada3db523659 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.045503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0050a7-4fc3-447c-a55d-ada3db523659 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.045781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0050a7-4fc3-447c-a55d-ada3db523659 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:46.046809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0050a7-4fc3-447c-a55d-ada3db523659 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:46.048232 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0050a7-4fc3-447c-a55d-ada3db523659 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:46.064039 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 480/975] 10.222.0.58 () {60 vars in 1141 bytes} [Wed Aug 24 12:20:45 2022] POST /identity/v3/services => generated 242 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:46.107737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3b3c51-5193-4279-84ba-1a9ec4307f06 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:46.158305 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0b31a-e8e2-476b-ade6-e7a17a0e1cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:46.158928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0b31a-e8e2-476b-ade6-e7a17a0e1cfd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.159211 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0b31a-e8e2-476b-ade6-e7a17a0e1cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.159484 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0b31a-e8e2-476b-ade6-e7a17a0e1cfd None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:46.160258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0b31a-e8e2-476b-ade6-e7a17a0e1cfd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:46.160584 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0b31a-e8e2-476b-ade6-e7a17a0e1cfd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:46.162022 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0b31a-e8e2-476b-ade6-e7a17a0e1cfd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:46.187711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 496/976] 10.222.0.58 () {56 vars in 1149 bytes} [Wed Aug 24 12:20:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:46.189962 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0050a7-4fc3-447c-a55d-ada3db523659 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:46.300048 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0050a7-4fc3-447c-a55d-ada3db523659 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:46.328075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:46.338966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841827c-5d2a-407e-9f07-e235306816f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:46.339584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841827c-5d2a-407e-9f07-e235306816f3 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.339900 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841827c-5d2a-407e-9f07-e235306816f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.340175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841827c-5d2a-407e-9f07-e235306816f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:46.410249 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841827c-5d2a-407e-9f07-e235306816f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:46.411726 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841827c-5d2a-407e-9f07-e235306816f3 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:46.517678 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 481/977] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:46 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:20:46.518852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0b31a-e8e2-476b-ade6-e7a17a0e1cfd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:46.606626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:46.625263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0b31a-e8e2-476b-ade6-e7a17a0e1cfd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:46.659342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308159cb-cdff-4d5f-8a8c-560c0aae3080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:46.660014 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308159cb-cdff-4d5f-8a8c-560c0aae3080 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.660321 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308159cb-cdff-4d5f-8a8c-560c0aae3080 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.660579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308159cb-cdff-4d5f-8a8c-560c0aae3080 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:46.671770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308159cb-cdff-4d5f-8a8c-560c0aae3080 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:46.673229 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308159cb-cdff-4d5f-8a8c-560c0aae3080 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:46.688631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-308159cb-cdff-4d5f-8a8c-560c0aae3080 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:46.688631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:46.688631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:46.688631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:46.688631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:46.688631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:46.688631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:46.688631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:46.688631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:46.688631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:46.688631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:46.688631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:46.688631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:46.688631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:46.688631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:46.688631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:46.689742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:46.690671 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:46.690671 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:46.690671 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:46.690671 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 497/978] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:46.696698 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841827c-5d2a-407e-9f07-e235306816f3 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:46.728416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:46.732160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140751a-9ade-446b-8c70-0c594f1a35c4 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.732716 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140751a-9ade-446b-8c70-0c594f1a35c4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.732824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140751a-9ade-446b-8c70-0c594f1a35c4 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:46.734347 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 498/979] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:46.740574 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b8881-5304-412e-a41c-79265f46e29e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.741056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b8881-5304-412e-a41c-79265f46e29e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.741938 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b8881-5304-412e-a41c-79265f46e29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:46.762518 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e71b8881-5304-412e-a41c-79265f46e29e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:46.764309 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e71b8881-5304-412e-a41c-79265f46e29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:46.802192 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841827c-5d2a-407e-9f07-e235306816f3 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:46.832468 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 499/980] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:46.837470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6ddb1-460f-4d51-9618-58832a177598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:46.838175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6ddb1-460f-4d51-9618-58832a177598 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.838454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6ddb1-460f-4d51-9618-58832a177598 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.838726 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6ddb1-460f-4d51-9618-58832a177598 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:46.839366 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6ddb1-460f-4d51-9618-58832a177598 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:46.839694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6ddb1-460f-4d51-9618-58832a177598 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:46.841061 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6ddb1-460f-4d51-9618-58832a177598 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:46.857204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 482/981] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:46.858378 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b8881-5304-412e-a41c-79265f46e29e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:46.928264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:46.963954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b8881-5304-412e-a41c-79265f46e29e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:46.997643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b710d-90dd-4e5f-9220-a85428969eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:46.998261 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b710d-90dd-4e5f-9220-a85428969eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:46.998540 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b710d-90dd-4e5f-9220-a85428969eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:46.998868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b710d-90dd-4e5f-9220-a85428969eb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:47.005476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b710d-90dd-4e5f-9220-a85428969eb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:47.006913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0b710d-90dd-4e5f-9220-a85428969eb0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:47.017915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-5f0b710d-90dd-4e5f-9220-a85428969eb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:47.017915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:47.017915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:47.017915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:47.017915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:47.017915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:47.017915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:47.017915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:47.017915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:47.017915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:47.017915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:47.017915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:47.017915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:47.017915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:47.017915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:47.018811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:47.025195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:47.025195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:47.025195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:47.025195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:47.025195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:47.025195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:47.025195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:47.025195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:47.025195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 500/982] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:47.027704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6ddb1-460f-4d51-9618-58832a177598 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:47.116181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b710d-90dd-4e5f-9220-a85428969eb0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:47.120247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:47.135222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6ddb1-460f-4d51-9618-58832a177598 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:47.171899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f94449-2d64-454f-b302-61cb52142e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:47.172517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f94449-2d64-454f-b302-61cb52142e79 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:47.172804 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f94449-2d64-454f-b302-61cb52142e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:47.173080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f94449-2d64-454f-b302-61cb52142e79 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:47.173783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f94449-2d64-454f-b302-61cb52142e79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:47.174090 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f94449-2d64-454f-b302-61cb52142e79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:47.175534 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f94449-2d64-454f-b302-61cb52142e79 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:47.189652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 483/983] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:47.195535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f94449-2d64-454f-b302-61cb52142e79 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:47.239836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b710d-90dd-4e5f-9220-a85428969eb0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:47.267343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:47.278116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094aa71-13af-497c-8cc3-ca7852106ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:47.278768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094aa71-13af-497c-8cc3-ca7852106ccd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:47.279050 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094aa71-13af-497c-8cc3-ca7852106ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:47.279324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094aa71-13af-497c-8cc3-ca7852106ccd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:47.279958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094aa71-13af-497c-8cc3-ca7852106ccd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:47.281547 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094aa71-13af-497c-8cc3-ca7852106ccd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-9094aa71-13af-497c-8cc3-ca7852106ccd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:47.291510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:47.292703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:47.292703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:47.292703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:47.292703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:47.292703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:47.292703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:47.292703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:47.292703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:47.292703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:47.292703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:47.292703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:47.292703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:47.292703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:47.292703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:47.292703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:47.292703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:47.293567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:47.293567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:47.293567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:47.293567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:47.293567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:47.293567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:47.293567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:47.293567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:47.293567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:47.293567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:47.293567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:47.293567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:47.293567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:47.293567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:47.293567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 501/984] 10.222.0.58 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:47.298559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094aa71-13af-497c-8cc3-ca7852106ccd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:47.299957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f94449-2d64-454f-b302-61cb52142e79 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:47.335823 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd999fa9-d75a-48d1-bb10-230966b3ab15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:47.336432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd999fa9-d75a-48d1-bb10-230966b3ab15 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:47.336715 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd999fa9-d75a-48d1-bb10-230966b3ab15 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:47.337011 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd999fa9-d75a-48d1-bb10-230966b3ab15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:47.338403 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd999fa9-d75a-48d1-bb10-230966b3ab15 None admin] RBAC: Adding query filter params (role.id=4c5d21de669d42169b9218cb54f295e4, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=adc6a2b3f6bd45178cc7ffb181c3c016) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:47.338715 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd999fa9-d75a-48d1-bb10-230966b3ab15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:47.340175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd999fa9-d75a-48d1-bb10-230966b3ab15 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:47.347673 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 484/985] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:47 2022] GET /identity/v3/role_assignments?user.id=adc6a2b3f6bd45178cc7ffb181c3c016&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=4c5d21de669d42169b9218cb54f295e4 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:47.408614 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094aa71-13af-497c-8cc3-ca7852106ccd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:47.444780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e70d6-2d27-4bf0-802f-608fbddfdec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:47.445472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e70d6-2d27-4bf0-802f-608fbddfdec0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:47.445757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e70d6-2d27-4bf0-802f-608fbddfdec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:47.446031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e70d6-2d27-4bf0-802f-608fbddfdec0 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:47.447221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e70d6-2d27-4bf0-802f-608fbddfdec0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:47.447521 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e70d6-2d27-4bf0-802f-608fbddfdec0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:47.449087 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e70d6-2d27-4bf0-802f-608fbddfdec0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:47.459692 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 502/986] 10.222.0.58 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:47.460730 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd999fa9-d75a-48d1-bb10-230966b3ab15 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:47.477823 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:47.574596 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd999fa9-d75a-48d1-bb10-230966b3ab15 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:47.609221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ed7df-bdeb-4e4b-aecf-b74b5f564f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:47.609836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ed7df-bdeb-4e4b-aecf-b74b5f564f8c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:47.610120 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ed7df-bdeb-4e4b-aecf-b74b5f564f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:47.610396 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ed7df-bdeb-4e4b-aecf-b74b5f564f8c None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:47.611027 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ed7df-bdeb-4e4b-aecf-b74b5f564f8c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:47.611333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ed7df-bdeb-4e4b-aecf-b74b5f564f8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:47.612774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ed7df-bdeb-4e4b-aecf-b74b5f564f8c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:47.622513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 485/987] 10.222.0.58 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:47.623816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e70d6-2d27-4bf0-802f-608fbddfdec0 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:47.681765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:47.734383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e70d6-2d27-4bf0-802f-608fbddfdec0 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:47.769410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a972bb-73ab-43a2-ab29-b3864d8c9047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:47.770311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a972bb-73ab-43a2-ab29-b3864d8c9047 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:47.770776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a972bb-73ab-43a2-ab29-b3864d8c9047 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:47.771232 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a972bb-73ab-43a2-ab29-b3864d8c9047 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:47.772176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a972bb-73ab-43a2-ab29-b3864d8c9047 None admin] RBAC: Adding query filter params (interface=admin, service_id=cab6accadd1d4d9fa5a1830be2fa5203, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:47.772670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a972bb-73ab-43a2-ab29-b3864d8c9047 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:47.774213 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a972bb-73ab-43a2-ab29-b3864d8c9047 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:47.780369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 503/988] 10.222.0.58 () {56 vars in 1249 bytes} [Wed Aug 24 12:20:47 2022] GET /identity/v3/endpoints?service_id=cab6accadd1d4d9fa5a1830be2fa5203&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:47.785681 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ed7df-bdeb-4e4b-aecf-b74b5f564f8c None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:47.868952 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:47.897057 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ed7df-bdeb-4e4b-aecf-b74b5f564f8c None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:47.932922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc04d484-b6b9-47a9-9bc1-f89726cc56f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:47.933450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04d484-b6b9-47a9-9bc1-f89726cc56f9 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:47.933741 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04d484-b6b9-47a9-9bc1-f89726cc56f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:47.934098 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc04d484-b6b9-47a9-9bc1-f89726cc56f9 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:47.934716 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04d484-b6b9-47a9-9bc1-f89726cc56f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:47.936122 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc04d484-b6b9-47a9-9bc1-f89726cc56f9 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:47.942704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 486/989] 10.222.0.58 () {56 vars in 1087 bytes} [Wed Aug 24 12:20:47 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:20:48.014979 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22918d73-96a9-4e06-a912-cc3f051b9bbc None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:48.015340 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22918d73-96a9-4e06-a912-cc3f051b9bbc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:48.015540 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22918d73-96a9-4e06-a912-cc3f051b9bbc None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:48.017186 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 504/990] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:48.022876 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0df34b-ec7b-449e-8201-e4c62e8ef8c1 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:48.023196 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0df34b-ec7b-449e-8201-e4c62e8ef8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:48.023441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0df34b-ec7b-449e-8201-e4c62e8ef8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:48.044886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-2f0df34b-ec7b-449e-8201-e4c62e8ef8c1 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:48.046113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2f0df34b-ec7b-449e-8201-e4c62e8ef8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:48.120894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 487/991] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:48.397011 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22918d73-96a9-4e06-a912-cc3f051b9bbc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:48.507324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22918d73-96a9-4e06-a912-cc3f051b9bbc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:48.525057 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:48.543438 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f254cb-6c91-493a-8443-99aa18bc73ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:48.544100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f254cb-6c91-493a-8443-99aa18bc73ec None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:48.544384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f254cb-6c91-493a-8443-99aa18bc73ec None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:48.544660 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f254cb-6c91-493a-8443-99aa18bc73ec None admin] PATH_INFO: `/v3/services/compute` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:48.545239 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f254cb-6c91-493a-8443-99aa18bc73ec None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:48.546609 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f254cb-6c91-493a-8443-99aa18bc73ec None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-36f254cb-6c91-493a-8443-99aa18bc73ec None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:48.555377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:48.557031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:48.557031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:48.557031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:48.557031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:48.557031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:48.557031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:48.557031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:48.557031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:48.557031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:48.557031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:48.557031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:48.557031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:48.557031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:48.557031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:48.557031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:48.557031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:48.557873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:48.557873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:48.557873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:48.557873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:48.557873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:48.557873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:48.557873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:48.557873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:48.557873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:48.557873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:48.557873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:48.557873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:48.557873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:48.557873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:48.559422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 505/992] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:48.562875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0df34b-ec7b-449e-8201-e4c62e8ef8c1 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:48.633014 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:48.666214 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0df34b-ec7b-449e-8201-e4c62e8ef8c1 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:48.701076 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6b917-af65-4cdf-a10e-efce3813b42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:48.701686 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6b917-af65-4cdf-a10e-efce3813b42b None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:48.701966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6b917-af65-4cdf-a10e-efce3813b42b None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:48.702237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd6b917-af65-4cdf-a10e-efce3813b42b None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:48.702859 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6b917-af65-4cdf-a10e-efce3813b42b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:48.703160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6b917-af65-4cdf-a10e-efce3813b42b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:48.704549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd6b917-af65-4cdf-a10e-efce3813b42b None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:48.713920 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 488/993] 10.222.0.58 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:48.715505 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f254cb-6c91-493a-8443-99aa18bc73ec None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:48.819954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f254cb-6c91-493a-8443-99aa18bc73ec None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:48.855681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80227b-7cb0-47db-8944-6d74c2010016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:48.856295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80227b-7cb0-47db-8944-6d74c2010016 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:48.856583 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80227b-7cb0-47db-8944-6d74c2010016 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:48.856885 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80227b-7cb0-47db-8944-6d74c2010016 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:48.857491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80227b-7cb0-47db-8944-6d74c2010016 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:48.857798 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80227b-7cb0-47db-8944-6d74c2010016 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:48.859165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a80227b-7cb0-47db-8944-6d74c2010016 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:48.869484 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 506/994] 10.222.0.58 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:48.870982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6b917-af65-4cdf-a10e-efce3813b42b None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:48.977076 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd6b917-af65-4cdf-a10e-efce3813b42b None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:49.012707 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751de4b-9c63-4e87-806e-198bf45ac1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:49.013318 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751de4b-9c63-4e87-806e-198bf45ac1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.013601 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751de4b-9c63-4e87-806e-198bf45ac1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.013873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751de4b-9c63-4e87-806e-198bf45ac1b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:49.014567 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751de4b-9c63-4e87-806e-198bf45ac1b5 None admin] RBAC: Adding query filter params (interface=public, service_id=4f32bed999f64453a53b5b9cbc71eff9, region_id=RegionOne) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:49.014878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751de4b-9c63-4e87-806e-198bf45ac1b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:49.016278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751de4b-9c63-4e87-806e-198bf45ac1b5 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:49.017767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:49.022934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 489/995] 10.222.0.58 () {56 vars in 1251 bytes} [Wed Aug 24 12:20:48 2022] GET /identity/v3/endpoints?service_id=4f32bed999f64453a53b5b9cbc71eff9&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:49.028995 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80227b-7cb0-47db-8944-6d74c2010016 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:49.081489 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:49.138477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80227b-7cb0-47db-8944-6d74c2010016 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:49.173841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9073bcd1-604f-4b3c-a58f-1abe8d99855d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:49.174450 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073bcd1-604f-4b3c-a58f-1abe8d99855d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.174732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073bcd1-604f-4b3c-a58f-1abe8d99855d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.175006 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073bcd1-604f-4b3c-a58f-1abe8d99855d None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:49.175608 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073bcd1-604f-4b3c-a58f-1abe8d99855d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:49.177004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9073bcd1-604f-4b3c-a58f-1abe8d99855d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:49.184437 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 507/996] 10.222.0.58 () {56 vars in 1087 bytes} [Wed Aug 24 12:20:49 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:20:49.310998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c9f6e-941b-4f4a-b555-ab59352ed220 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.311374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c9f6e-941b-4f4a-b555-ab59352ed220 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.311550 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c9f6e-941b-4f4a-b555-ab59352ed220 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:49.313192 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 490/997] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:49.318634 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bdb70-4c01-4257-a88b-77e70f079a02 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.318924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bdb70-4c01-4257-a88b-77e70f079a02 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.319197 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223bdb70-4c01-4257-a88b-77e70f079a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:49.340318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-223bdb70-4c01-4257-a88b-77e70f079a02 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:49.341515 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-223bdb70-4c01-4257-a88b-77e70f079a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:49.410099 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 508/998] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:49.684072 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c9f6e-941b-4f4a-b555-ab59352ed220 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:49.789855 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c9f6e-941b-4f4a-b555-ab59352ed220 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:49.824306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315845e-8512-4a0d-b05d-2a1c74f7f63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:49.825222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315845e-8512-4a0d-b05d-2a1c74f7f63a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.825502 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315845e-8512-4a0d-b05d-2a1c74f7f63a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.825775 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315845e-8512-4a0d-b05d-2a1c74f7f63a None admin] PATH_INFO: `/v3/services/image` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:49.826355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315845e-8512-4a0d-b05d-2a1c74f7f63a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:49.827757 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315845e-8512-4a0d-b05d-2a1c74f7f63a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-c315845e-8512-4a0d-b05d-2a1c74f7f63a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:49.836513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:49.838153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:49.838153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:49.838153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:49.838153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:49.838153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:49.838153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:49.838153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:49.838153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:49.838153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:49.838153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:49.838153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:49.838153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:49.838153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:49.838153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:49.838153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:49.838153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:49.840175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:49.840175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:49.840175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:49.840175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:49.840175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:49.840175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:49.840175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:49.840175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:49.840175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:49.840175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:49.840175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:49.840175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:49.840175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:20:49.840175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:49.840175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 491/999] 10.222.0.58 () {56 vars in 1105 bytes} [Wed Aug 24 12:20:49 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:49.844264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223bdb70-4c01-4257-a88b-77e70f079a02 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:49.889972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:49.904975 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a08a4-ae9b-43d7-9b2f-a4597440d861 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.905475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a08a4-ae9b-43d7-9b2f-a4597440d861 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.905672 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a08a4-ae9b-43d7-9b2f-a4597440d861 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:49.907152 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 492/1000] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:49.912390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d60f49-3cd9-4d8a-bb63-ebdac4b71dc3 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:49.912620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d60f49-3cd9-4d8a-bb63-ebdac4b71dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:49.914037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d60f49-3cd9-4d8a-bb63-ebdac4b71dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:49.931781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-37d60f49-3cd9-4d8a-bb63-ebdac4b71dc3 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:49.933062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-37d60f49-3cd9-4d8a-bb63-ebdac4b71dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:49.962229 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223bdb70-4c01-4257-a88b-77e70f079a02 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:49.998018 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:50.007445 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44454817-0560-4910-bece-705ec02cca13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:50.008085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44454817-0560-4910-bece-705ec02cca13 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:50.008368 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44454817-0560-4910-bece-705ec02cca13 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:50.008642 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44454817-0560-4910-bece-705ec02cca13 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.009873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44454817-0560-4910-bece-705ec02cca13 None admin] RBAC: Adding query filter params (name=image) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:50.010178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44454817-0560-4910-bece-705ec02cca13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:50.010766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 493/1001] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:50.011551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44454817-0560-4910-bece-705ec02cca13 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:50.016799 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 509/1002] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:50.022186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d60f49-3cd9-4d8a-bb63-ebdac4b71dc3 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:50.132966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d60f49-3cd9-4d8a-bb63-ebdac4b71dc3 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:50.167119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b273b13-d049-457a-beca-d7279e8dc3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:50.167765 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b273b13-d049-457a-beca-d7279e8dc3be None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:50.168051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b273b13-d049-457a-beca-d7279e8dc3be None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:50.168325 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b273b13-d049-457a-beca-d7279e8dc3be None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.168961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b273b13-d049-457a-beca-d7279e8dc3be None admin] RBAC: Adding query filter params (type=image) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:50.169269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b273b13-d049-457a-beca-d7279e8dc3be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:50.170634 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b273b13-d049-457a-beca-d7279e8dc3be None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:50.175205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 494/1003] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:50.287010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44454817-0560-4910-bece-705ec02cca13 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:50.392769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44454817-0560-4910-bece-705ec02cca13 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:50.395135 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:50.429222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee312f01-f7c3-4243-8828-884a8e847f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:50.429865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee312f01-f7c3-4243-8828-884a8e847f88 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:50.430136 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee312f01-f7c3-4243-8828-884a8e847f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:50.430410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee312f01-f7c3-4243-8828-884a8e847f88 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.431047 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee312f01-f7c3-4243-8828-884a8e847f88 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:50.432441 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee312f01-f7c3-4243-8828-884a8e847f88 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-ee312f01-f7c3-4243-8828-884a8e847f88 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:50.441078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:50.442298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:50.442298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:50.442298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:50.442298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:50.442298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:50.442298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:50.442298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:50.442298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:50.442298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:50.442298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:50.442298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:50.442298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:50.442298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:50.442298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:50.442298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:50.442298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:50.443168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:50.443168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:50.443168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:50.443168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:50.443168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:50.443168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:50.443168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:50.443168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:50.443168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:50.443168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:50.443168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:50.443168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:50.443168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 12:20:50.443168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:50.447610 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 510/1004] 10.222.0.58 () {56 vars in 1126 bytes} [Wed Aug 24 12:20:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:50.448995 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b273b13-d049-457a-beca-d7279e8dc3be None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:50.506746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:50.551465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b273b13-d049-457a-beca-d7279e8dc3be None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:50.586087 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446466f-92a1-49a5-894c-461d670302da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:50.586701 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446466f-92a1-49a5-894c-461d670302da None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:50.587542 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446466f-92a1-49a5-894c-461d670302da None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:50.587843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446466f-92a1-49a5-894c-461d670302da None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.588469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446466f-92a1-49a5-894c-461d670302da None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:50.588770 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446466f-92a1-49a5-894c-461d670302da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:50.590139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446466f-92a1-49a5-894c-461d670302da None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:50.599806 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 495/1005] 10.222.0.58 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:50.600754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee312f01-f7c3-4243-8828-884a8e847f88 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:50.704698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee312f01-f7c3-4243-8828-884a8e847f88 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:50.738237 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916680b6-9f7c-4bf9-93d0-ecf36a1be575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:50.738846 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916680b6-9f7c-4bf9-93d0-ecf36a1be575 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:50.739130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916680b6-9f7c-4bf9-93d0-ecf36a1be575 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:50.739401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916680b6-9f7c-4bf9-93d0-ecf36a1be575 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.740054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916680b6-9f7c-4bf9-93d0-ecf36a1be575 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:50.740360 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916680b6-9f7c-4bf9-93d0-ecf36a1be575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:50.741733 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916680b6-9f7c-4bf9-93d0-ecf36a1be575 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:50.753309 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 511/1006] 10.222.0.58 () {56 vars in 1122 bytes} [Wed Aug 24 12:20:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:50.753466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446466f-92a1-49a5-894c-461d670302da None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:50.852407 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446466f-92a1-49a5-894c-461d670302da None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:50.872836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:50.897158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8cb87-de9b-4173-ba6e-05f09dbb6102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:50.897774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8cb87-de9b-4173-ba6e-05f09dbb6102 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:50.898063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8cb87-de9b-4173-ba6e-05f09dbb6102 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:50.898412 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8cb87-de9b-4173-ba6e-05f09dbb6102 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:50.899081 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8cb87-de9b-4173-ba6e-05f09dbb6102 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:50.900774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8cb87-de9b-4173-ba6e-05f09dbb6102 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:50.923648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 496/1007] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:20:50 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:20:51.145965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f80a1-2451-4e91-bfbc-6884e1f9a89c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:51.146458 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f80a1-2451-4e91-bfbc-6884e1f9a89c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:51.146458 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28f80a1-2451-4e91-bfbc-6884e1f9a89c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:51.147814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 512/1008] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:51.153469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fcbd3-7a0b-432e-8eba-a77e16428ed9 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:51.153763 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fcbd3-7a0b-432e-8eba-a77e16428ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:51.154040 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fcbd3-7a0b-432e-8eba-a77e16428ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:51.172504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-af5fcbd3-7a0b-432e-8eba-a77e16428ed9 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:51.173698 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-af5fcbd3-7a0b-432e-8eba-a77e16428ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:51.248852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 497/1009] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:51 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:51.519132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f80a1-2451-4e91-bfbc-6884e1f9a89c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:51.550870 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:51.624644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28f80a1-2451-4e91-bfbc-6884e1f9a89c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:51.658269 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba7349-2a4e-4620-8ffb-16ae859d860f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:51.658890 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba7349-2a4e-4620-8ffb-16ae859d860f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:51.659169 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba7349-2a4e-4620-8ffb-16ae859d860f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:51.659440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba7349-2a4e-4620-8ffb-16ae859d860f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:51.660051 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba7349-2a4e-4620-8ffb-16ae859d860f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:51.661418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba7349-2a4e-4620-8ffb-16ae859d860f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-ddba7349-2a4e-4620-8ffb-16ae859d860f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:51.670187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:51.671161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:51.671161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:51.671161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:51.671161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:51.671161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:51.671161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:51.671161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:51.671161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:51.671161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:51.671161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:51.671161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:51.671161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:51.671161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:51.671161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:51.671161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:51.671161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:51.672401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:51.672401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:51.672401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:51.672401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:51.672401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:51.672401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:51.672401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:51.672401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:51.672401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:51.672401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:51.672401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:51.672401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:51.672401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 12:20:51.672401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:51.675935 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 513/1010] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:51.677261 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fcbd3-7a0b-432e-8eba-a77e16428ed9 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:51.777230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fcbd3-7a0b-432e-8eba-a77e16428ed9 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:51.784748 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:51.814218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc28275-aa91-484c-b14b-442298b36f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:51.814833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc28275-aa91-484c-b14b-442298b36f3c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:51.815105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc28275-aa91-484c-b14b-442298b36f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:51.815375 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc28275-aa91-484c-b14b-442298b36f3c None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:51.816032 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc28275-aa91-484c-b14b-442298b36f3c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:51.816339 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc28275-aa91-484c-b14b-442298b36f3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:51.817704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc28275-aa91-484c-b14b-442298b36f3c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:51.827994 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 498/1011] 10.222.0.58 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:51.828836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba7349-2a4e-4620-8ffb-16ae859d860f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:51.934628 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:51.941430 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba7349-2a4e-4620-8ffb-16ae859d860f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:51.982425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb3f53-6d30-4bce-84a5-784133e60594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:51.983028 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb3f53-6d30-4bce-84a5-784133e60594 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:51.983304 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb3f53-6d30-4bce-84a5-784133e60594 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:51.983573 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb3f53-6d30-4bce-84a5-784133e60594 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:51.984220 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb3f53-6d30-4bce-84a5-784133e60594 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:51.984521 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb3f53-6d30-4bce-84a5-784133e60594 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:51.985879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb3f53-6d30-4bce-84a5-784133e60594 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:51.996313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 514/1012] 10.222.0.58 () {56 vars in 1112 bytes} [Wed Aug 24 12:20:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:51.998046 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc28275-aa91-484c-b14b-442298b36f3c None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:52.102374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc28275-aa91-484c-b14b-442298b36f3c None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:52.111316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffb4cf-dd18-46af-8f49-dd988912fbbf None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:52.111612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffb4cf-dd18-46af-8f49-dd988912fbbf None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:52.111923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ffb4cf-dd18-46af-8f49-dd988912fbbf None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:52.113419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 515/1013] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:52.118090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c297b3-29a2-4afd-902b-5fd81d387c25 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:52.118922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c297b3-29a2-4afd-902b-5fd81d387c25 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:52.119199 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c297b3-29a2-4afd-902b-5fd81d387c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:52.136674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-99c297b3-29a2-4afd-902b-5fd81d387c25 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:52.137881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-99c297b3-29a2-4afd-902b-5fd81d387c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:52.138079 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec0df1-ecd2-42b7-a66b-fccd2952cbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:52.138673 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec0df1-ecd2-42b7-a66b-fccd2952cbb7 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:52.138950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec0df1-ecd2-42b7-a66b-fccd2952cbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:52.139219 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec0df1-ecd2-42b7-a66b-fccd2952cbb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:52.139904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec0df1-ecd2-42b7-a66b-fccd2952cbb7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:52.141250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec0df1-ecd2-42b7-a66b-fccd2952cbb7 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:52.171650 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 499/1014] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:20:51 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:20:52.230060 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 516/1015] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:52 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:52.495853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec0df1-ecd2-42b7-a66b-fccd2952cbb7 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:52.523689 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:52.600511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec0df1-ecd2-42b7-a66b-fccd2952cbb7 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:52.633684 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bcac5e-894e-4115-a877-9ab4f31f642c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:52.634818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bcac5e-894e-4115-a877-9ab4f31f642c None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:52.635095 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bcac5e-894e-4115-a877-9ab4f31f642c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:52.635363 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bcac5e-894e-4115-a877-9ab4f31f642c None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:52.636036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bcac5e-894e-4115-a877-9ab4f31f642c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:52.637404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bcac5e-894e-4115-a877-9ab4f31f642c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:52.655677 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 500/1016] 10.222.0.58 () {60 vars in 1141 bytes} [Wed Aug 24 12:20:52 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:52.852710 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889e839-79d9-484e-9b1c-7dfdaaf1077a None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:52.853085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889e839-79d9-484e-9b1c-7dfdaaf1077a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:52.853328 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889e839-79d9-484e-9b1c-7dfdaaf1077a None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:52.854761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 517/1017] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:52.859815 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419aa40-8e55-4687-a30a-3f9c3a1e5439 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:52.860105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419aa40-8e55-4687-a30a-3f9c3a1e5439 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:52.860382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419aa40-8e55-4687-a30a-3f9c3a1e5439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:52.886180 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-d419aa40-8e55-4687-a30a-3f9c3a1e5439 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:52.887404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d419aa40-8e55-4687-a30a-3f9c3a1e5439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:52.964069 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 501/1018] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:53.234396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889e839-79d9-484e-9b1c-7dfdaaf1077a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:53.327323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:53.344958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889e839-79d9-484e-9b1c-7dfdaaf1077a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:53.380416 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed302bb-a412-49e4-a7f8-acbde6b1309d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:53.381048 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed302bb-a412-49e4-a7f8-acbde6b1309d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:53.381331 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed302bb-a412-49e4-a7f8-acbde6b1309d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:53.381605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed302bb-a412-49e4-a7f8-acbde6b1309d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:53.382721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed302bb-a412-49e4-a7f8-acbde6b1309d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:53.384956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed302bb-a412-49e4-a7f8-acbde6b1309d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:53.386721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 518/1019] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:53.392486 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419aa40-8e55-4687-a30a-3f9c3a1e5439 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:53.407068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:53.492958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419aa40-8e55-4687-a30a-3f9c3a1e5439 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:53.528455 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d4d6f-7e6a-4d22-aca4-185782a2f75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:53.529066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d4d6f-7e6a-4d22-aca4-185782a2f75f None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:53.529348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d4d6f-7e6a-4d22-aca4-185782a2f75f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:53.529621 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d4d6f-7e6a-4d22-aca4-185782a2f75f None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:53.530269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d4d6f-7e6a-4d22-aca4-185782a2f75f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:53.531739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d4d6f-7e6a-4d22-aca4-185782a2f75f None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:53.567658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 502/1020] 10.222.0.58 () {60 vars in 1141 bytes} [Wed Aug 24 12:20:53 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:54.217150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b23f95-768b-4fa9-b937-8ee72c8b4afc None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:54.217450 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b23f95-768b-4fa9-b937-8ee72c8b4afc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:54.217630 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b23f95-768b-4fa9-b937-8ee72c8b4afc None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:54.219153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 519/1021] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:54.224241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495c66a-a5a0-4fe2-9802-cf6057d9ae20 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:54.224534 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495c66a-a5a0-4fe2-9802-cf6057d9ae20 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:54.224812 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a495c66a-a5a0-4fe2-9802-cf6057d9ae20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:54.243836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-a495c66a-a5a0-4fe2-9802-cf6057d9ae20 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:54.245040 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a495c66a-a5a0-4fe2-9802-cf6057d9ae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:54.305593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:54.319708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 503/1022] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:54.585218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b23f95-768b-4fa9-b937-8ee72c8b4afc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:54.592623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fed92b-fc6d-4eef-b76c-ee73a07d5159 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:54.593079 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fed92b-fc6d-4eef-b76c-ee73a07d5159 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:54.593157 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fed92b-fc6d-4eef-b76c-ee73a07d5159 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:54.594662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 504/1023] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:54.599356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4b6a0-6c51-4415-b44b-bad0c99d449c None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:54.599648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4b6a0-6c51-4415-b44b-bad0c99d449c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:54.599926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4b6a0-6c51-4415-b44b-bad0c99d449c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:54.618024 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-45e4b6a0-6c51-4415-b44b-bad0c99d449c None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:54.619207 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-45e4b6a0-6c51-4415-b44b-bad0c99d449c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:54.703854 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 505/1024] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:54.704070 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b23f95-768b-4fa9-b937-8ee72c8b4afc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:54.740657 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728b96f-0d82-4bf3-8cda-f11c2de22459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:54.741565 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728b96f-0d82-4bf3-8cda-f11c2de22459 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:54.742019 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728b96f-0d82-4bf3-8cda-f11c2de22459 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:54.742442 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3728b96f-0d82-4bf3-8cda-f11c2de22459 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:54.752869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3728b96f-0d82-4bf3-8cda-f11c2de22459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:54.757800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-3728b96f-0d82-4bf3-8cda-f11c2de22459 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:20:54.757800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:54.757800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:54.757800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:54.757800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:54.757800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:54.757800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:54.757800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:54.757800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:54.757800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:54.757800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:54.757800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:54.757800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:54.757800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:54.757800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:54.757800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:54.759090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:54.759090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:54.759090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:54.759090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:54.759090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:54.759090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:54.759090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:54.759090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:54.759090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:54.759090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:54.759090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:54.759090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:54.759090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:54.759090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:54.759090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:54.759090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:54.760151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:54.760151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:54.760151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:54.760151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:54.760151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:54.760151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:54.760151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:54.760151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:54.760151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:54.760151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:54.760151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:20:54.760151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:54.767194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 520/1025] 10.222.0.58 () {56 vars in 1120 bytes} [Wed Aug 24 12:20:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:54.768302 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4b6a0-6c51-4415-b44b-bad0c99d449c None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:54.868943 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4b6a0-6c51-4415-b44b-bad0c99d449c None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:54.899110 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:54.911228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37005402-ff2d-4fab-b9a0-4df697379639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:54.911867 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37005402-ff2d-4fab-b9a0-4df697379639 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:54.912147 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37005402-ff2d-4fab-b9a0-4df697379639 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:54.912415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37005402-ff2d-4fab-b9a0-4df697379639 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:54.913070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37005402-ff2d-4fab-b9a0-4df697379639 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:54.913372 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37005402-ff2d-4fab-b9a0-4df697379639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:54.914724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37005402-ff2d-4fab-b9a0-4df697379639 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:54.927777 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 506/1026] 10.222.0.58 () {56 vars in 1115 bytes} [Wed Aug 24 12:20:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:54.928702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728b96f-0d82-4bf3-8cda-f11c2de22459 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:54.945616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:54.974123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37005402-ff2d-4fab-b9a0-4df697379639 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:55.049088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3728b96f-0d82-4bf3-8cda-f11c2de22459 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:55.080294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37005402-ff2d-4fab-b9a0-4df697379639 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:55.083532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284ab45-e343-430b-a920-1b18df8b7c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:55.084195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ab45-e343-430b-a920-1b18df8b7c43 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.084475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ab45-e343-430b-a920-1b18df8b7c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.084843 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ab45-e343-430b-a920-1b18df8b7c43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.098068 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284ab45-e343-430b-a920-1b18df8b7c43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:55.099522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e284ab45-e343-430b-a920-1b18df8b7c43 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:55.114976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e284ab45-e343-430b-a920-1b18df8b7c43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:55.114976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:55.114976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:55.114976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:55.114976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:55.114976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:55.114976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:55.114976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:55.114976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:55.114976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:55.114976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:55.114976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:55.114976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:55.114976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:55.114976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:55.114976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:55.116259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:55.117175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:55.117175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:55.117175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:55.117175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 521/1027] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:55.122270 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284ab45-e343-430b-a920-1b18df8b7c43 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:55.125772 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f0c10-fb03-475f-b625-9572a633af11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:55.126476 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f0c10-fb03-475f-b625-9572a633af11 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.126759 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f0c10-fb03-475f-b625-9572a633af11 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.127036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f0c10-fb03-475f-b625-9572a633af11 None admin] PATH_INFO: `/v3/services/image` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.127642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f0c10-fb03-475f-b625-9572a633af11 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:55.129016 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f0c10-fb03-475f-b625-9572a633af11 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-b96f0c10-fb03-475f-b625-9572a633af11 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:55.138608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:55.139662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:55.139662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:55.139662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:55.139662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:55.139662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:55.139662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:55.139662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:55.139662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:55.139662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:55.139662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:55.139662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:55.139662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:55.139662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:55.139662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:55.139662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:55.139662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:55.140538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:55.140538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:55.140538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:55.140538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:55.140538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:55.140538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:55.140538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:55.140538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:55.140538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:55.140538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:55.140538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:55.140538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:55.140538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:20:55.140538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:55.140538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 507/1028] 10.222.0.58 () {56 vars in 1105 bytes} [Wed Aug 24 12:20:54 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:55.145441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f0c10-fb03-475f-b625-9572a633af11 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:55.199051 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:55.202638 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:55.238901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284ab45-e343-430b-a920-1b18df8b7c43 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:55.257362 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f0c10-fb03-475f-b625-9572a633af11 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:55.273559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c870710-bc68-4ae1-a24b-19400351512e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:55.274167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c870710-bc68-4ae1-a24b-19400351512e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.274561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c870710-bc68-4ae1-a24b-19400351512e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.275036 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c870710-bc68-4ae1-a24b-19400351512e None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.275731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c870710-bc68-4ae1-a24b-19400351512e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:55.276041 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c870710-bc68-4ae1-a24b-19400351512e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:55.277406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c870710-bc68-4ae1-a24b-19400351512e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:55.289392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 522/1029] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:55.294968 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c870710-bc68-4ae1-a24b-19400351512e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:55.300069 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7873471-4255-43ec-b418-05a9558d0d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:55.300682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7873471-4255-43ec-b418-05a9558d0d17 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.300964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7873471-4255-43ec-b418-05a9558d0d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.301239 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7873471-4255-43ec-b418-05a9558d0d17 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.301902 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7873471-4255-43ec-b418-05a9558d0d17 None admin] RBAC: Adding query filter params (name=image) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:55.302212 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7873471-4255-43ec-b418-05a9558d0d17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:55.303578 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7873471-4255-43ec-b418-05a9558d0d17 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:55.309439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 508/1030] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:55.315120 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7873471-4255-43ec-b418-05a9558d0d17 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:55.401694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c870710-bc68-4ae1-a24b-19400351512e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:55.418416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7873471-4255-43ec-b418-05a9558d0d17 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:55.428689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:55.433397 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:55.439919 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b889a-dba4-4bb4-a872-4038b6217516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:55.440559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b889a-dba4-4bb4-a872-4038b6217516 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.440840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b889a-dba4-4bb4-a872-4038b6217516 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.441117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b889a-dba4-4bb4-a872-4038b6217516 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.449603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b889a-dba4-4bb4-a872-4038b6217516 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:55.451436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b889a-dba4-4bb4-a872-4038b6217516 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:55.455792 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0482e-34ea-49f5-b5f0-42d87f38ce87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:55.456408 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0482e-34ea-49f5-b5f0-42d87f38ce87 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.456690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0482e-34ea-49f5-b5f0-42d87f38ce87 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.456962 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0482e-34ea-49f5-b5f0-42d87f38ce87 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.457592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0482e-34ea-49f5-b5f0-42d87f38ce87 None admin] RBAC: Adding query filter params (type=image) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:55.457899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0482e-34ea-49f5-b5f0-42d87f38ce87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:55.459255 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0482e-34ea-49f5-b5f0-42d87f38ce87 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:55.463452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-301b889a-dba4-4bb4-a872-4038b6217516 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:55.463452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:55.463452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:55.463452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:55.463452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:55.463452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:55.463452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:55.463452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:55.463452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:55.463452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:55.463452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:55.463452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:55.463452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:20:55.463452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:20:55.463452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:20:55.463452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:55.464475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:55.465452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:55.465452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:55.465452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:20:55.465452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:20:55.465452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:20:55.465452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:20:55.465452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:20:55.465452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:20:55.465452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:20:55.465452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:55.465452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 523/1031] 10.222.0.58 () {56 vars in 1093 bytes} [Wed Aug 24 12:20:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:55.469524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 509/1032] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:55.470392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b889a-dba4-4bb4-a872-4038b6217516 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:55.473321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0482e-34ea-49f5-b5f0-42d87f38ce87 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:55.579559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b889a-dba4-4bb4-a872-4038b6217516 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:55.585166 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0482e-34ea-49f5-b5f0-42d87f38ce87 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:55.613807 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a363b-d4b2-4c49-be36-9aa63a0d4b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:55.614415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a363b-d4b2-4c49-be36-9aa63a0d4b0e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.614699 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a363b-d4b2-4c49-be36-9aa63a0d4b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.614972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a363b-d4b2-4c49-be36-9aa63a0d4b0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.615700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a363b-d4b2-4c49-be36-9aa63a0d4b0e None admin] RBAC: Adding query filter params (interface=public, service_id=00c0403951f548549e2804243d552dc3, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:55.616010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a363b-d4b2-4c49-be36-9aa63a0d4b0e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:55.617366 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a363b-d4b2-4c49-be36-9aa63a0d4b0e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:55.620070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63b800-9d4e-43fd-ad39-57675e8564f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:55.620685 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63b800-9d4e-43fd-ad39-57675e8564f1 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.620966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63b800-9d4e-43fd-ad39-57675e8564f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.621241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63b800-9d4e-43fd-ad39-57675e8564f1 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.621971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf63b800-9d4e-43fd-ad39-57675e8564f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:55.622279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf63b800-9d4e-43fd-ad39-57675e8564f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:55.623406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 524/1033] 10.222.0.58 () {56 vars in 1251 bytes} [Wed Aug 24 12:20:55 2022] GET /identity/v3/endpoints?service_id=00c0403951f548549e2804243d552dc3&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:55.623719 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf63b800-9d4e-43fd-ad39-57675e8564f1 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:55.624624 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0195d-58f8-435b-bfbb-304355236255 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.624915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0195d-58f8-435b-bfbb-304355236255 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.625193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0195d-58f8-435b-bfbb-304355236255 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.627524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 525/1034] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:55.632121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0195d-58f8-435b-bfbb-304355236255 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:55.646836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 510/1035] 10.222.0.58 () {56 vars in 1133 bytes} [Wed Aug 24 12:20:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:55.646836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3e7bd-b9af-4c25-8383-2aab0e1d0291 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.646836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3e7bd-b9af-4c25-8383-2aab0e1d0291 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.646836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3e7bd-b9af-4c25-8383-2aab0e1d0291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.662652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-1fe3e7bd-b9af-4c25-8383-2aab0e1d0291 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:55.663811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1fe3e7bd-b9af-4c25-8383-2aab0e1d0291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:55.682967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:55.692834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:55.733278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 511/1036] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:55.734185 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3e7bd-b9af-4c25-8383-2aab0e1d0291 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:55.748405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0195d-58f8-435b-bfbb-304355236255 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:55.784475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01dc6e-5ff3-4710-ad0f-323686257b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:55.785131 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01dc6e-5ff3-4710-ad0f-323686257b7d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.785414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01dc6e-5ff3-4710-ad0f-323686257b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.785688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01dc6e-5ff3-4710-ad0f-323686257b7d None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.786286 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01dc6e-5ff3-4710-ad0f-323686257b7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:55.787751 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01dc6e-5ff3-4710-ad0f-323686257b7d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:55.794652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 526/1037] 10.222.0.58 () {56 vars in 1087 bytes} [Wed Aug 24 12:20:55 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:20:55.848317 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3e7bd-b9af-4c25-8383-2aab0e1d0291 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:55.894244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e8000-b667-4682-8568-15fea379f33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:55.894901 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e8000-b667-4682-8568-15fea379f33c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:55.895190 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e8000-b667-4682-8568-15fea379f33c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:55.895461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10e8000-b667-4682-8568-15fea379f33c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:55.934178 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:55.974361 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e8000-b667-4682-8568-15fea379f33c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:55.975853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10e8000-b667-4682-8568-15fea379f33c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:56.006132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01dc6e-5ff3-4710-ad0f-323686257b7d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:56.090145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 512/1038] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:55 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:20:56.095493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e8000-b667-4682-8568-15fea379f33c None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:56.136269 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01dc6e-5ff3-4710-ad0f-323686257b7d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:56.151687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:56.174123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:56.174342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa8f58-d934-45b0-855d-4c47fdd36bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:56.174881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa8f58-d934-45b0-855d-4c47fdd36bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:56.175240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa8f58-d934-45b0-855d-4c47fdd36bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:56.175433 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa8f58-d934-45b0-855d-4c47fdd36bf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:56.176579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa8f58-d934-45b0-855d-4c47fdd36bf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:56.178017 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa8f58-d934-45b0-855d-4c47fdd36bf7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:56.183665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 527/1039] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:56.188862 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa8f58-d934-45b0-855d-4c47fdd36bf7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:56.217690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10e8000-b667-4682-8568-15fea379f33c None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:56.253713 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc7da8-08ad-468c-ba77-0f7e624c9cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:56.254344 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc7da8-08ad-468c-ba77-0f7e624c9cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:56.254619 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc7da8-08ad-468c-ba77-0f7e624c9cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:56.254897 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc7da8-08ad-468c-ba77-0f7e624c9cf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:56.265377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc7da8-08ad-468c-ba77-0f7e624c9cf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:56.266822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc7da8-08ad-468c-ba77-0f7e624c9cf7 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:56.282289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-03bc7da8-08ad-468c-ba77-0f7e624c9cf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:56.282289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:56.282289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:56.282289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:56.282289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:56.282289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:56.282289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:56.282289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:56.282289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:56.282289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:56.282289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:56.282289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:56.282289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:56.282289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:56.282289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:56.282289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:56.283356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:56.284427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:56.284427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:56.284427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:56.284427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 513/1040] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:56.288934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc7da8-08ad-468c-ba77-0f7e624c9cf7 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:56.301143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa8f58-d934-45b0-855d-4c47fdd36bf7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:56.339886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0a155-65d3-4c96-aba2-fc810e28bebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:56.340643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0a155-65d3-4c96-aba2-fc810e28bebe None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:56.340923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0a155-65d3-4c96-aba2-fc810e28bebe None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:56.341194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a0a155-65d3-4c96-aba2-fc810e28bebe None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:56.341835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0a155-65d3-4c96-aba2-fc810e28bebe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:56.343254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a0a155-65d3-4c96-aba2-fc810e28bebe None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:56.373964 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 528/1041] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:20:56 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:56.398294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc7da8-08ad-468c-ba77-0f7e624c9cf7 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:56.419784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:56.440788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdfbbe-081b-43f9-b124-ca790440552a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:56.441381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdfbbe-081b-43f9-b124-ca790440552a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:56.441667 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdfbbe-081b-43f9-b124-ca790440552a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:56.442036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdfbbe-081b-43f9-b124-ca790440552a None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:56.442705 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdfbbe-081b-43f9-b124-ca790440552a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:56.443014 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdfbbe-081b-43f9-b124-ca790440552a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:56.444600 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bdfbbe-081b-43f9-b124-ca790440552a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:56.459567 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 514/1042] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:56.465918 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0a155-65d3-4c96-aba2-fc810e28bebe None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:56.509300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:56.574783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a0a155-65d3-4c96-aba2-fc810e28bebe None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:56.609288 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f2664-b2ce-4cfd-ba55-4d80242b77eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:56.609927 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f2664-b2ce-4cfd-ba55-4d80242b77eb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:56.610206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f2664-b2ce-4cfd-ba55-4d80242b77eb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:56.610477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993f2664-b2ce-4cfd-ba55-4d80242b77eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:56.617135 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f2664-b2ce-4cfd-ba55-4d80242b77eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:56.618674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993f2664-b2ce-4cfd-ba55-4d80242b77eb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:56.629729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-993f2664-b2ce-4cfd-ba55-4d80242b77eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:56.629729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:56.629729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:56.629729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:56.629729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:56.629729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:56.629729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:56.629729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:56.629729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:56.629729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:56.629729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:56.629729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:56.629729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:20:56.629729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:20:56.629729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:56.630653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:56.631564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:20:56.631564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:20:56.631564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:20:56.631564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:20:56.631564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:20:56.631564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:20:56.631564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:20:56.631564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:56.631564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 529/1043] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:20:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:56.637410 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdfbbe-081b-43f9-b124-ca790440552a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:56.739763 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdfbbe-081b-43f9-b124-ca790440552a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:56.775899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d19355-86d9-43b5-81c6-0282b18c7dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:56.776518 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d19355-86d9-43b5-81c6-0282b18c7dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:56.776797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d19355-86d9-43b5-81c6-0282b18c7dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:56.777069 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d19355-86d9-43b5-81c6-0282b18c7dc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:56.777764 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d19355-86d9-43b5-81c6-0282b18c7dc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:56.778070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d19355-86d9-43b5-81c6-0282b18c7dc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:56.779503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d19355-86d9-43b5-81c6-0282b18c7dc0 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:56.796333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 515/1044] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:20:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:56.797522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f2664-b2ce-4cfd-ba55-4d80242b77eb None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:56.901987 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:56.906146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993f2664-b2ce-4cfd-ba55-4d80242b77eb None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:56.941876 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fff849-dad8-435a-94e6-a37f8e7aa288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:56.942486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fff849-dad8-435a-94e6-a37f8e7aa288 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:56.942765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fff849-dad8-435a-94e6-a37f8e7aa288 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:56.943039 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fff849-dad8-435a-94e6-a37f8e7aa288 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:56.943801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fff849-dad8-435a-94e6-a37f8e7aa288 None admin] RBAC: Adding query filter params (role.id=4c5d21de669d42169b9218cb54f295e4, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=311dbed3ab0b44a7830a93227ff9ad33) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:56.944114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fff849-dad8-435a-94e6-a37f8e7aa288 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:56.945543 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fff849-dad8-435a-94e6-a37f8e7aa288 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:56.959907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 530/1045] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:56 2022] GET /identity/v3/role_assignments?user.id=311dbed3ab0b44a7830a93227ff9ad33&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=4c5d21de669d42169b9218cb54f295e4 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:57.729750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a0473-96e4-457a-b6b0-c8b215e1ebcb None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:57.730306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a0473-96e4-457a-b6b0-c8b215e1ebcb None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:57.730306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a0473-96e4-457a-b6b0-c8b215e1ebcb None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:57.731665 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 516/1046] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:20:57.736974 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713b96e-4f21-42b7-b33a-042452d0d6bb None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:57.737355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713b96e-4f21-42b7-b33a-042452d0d6bb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:57.737538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713b96e-4f21-42b7-b33a-042452d0d6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:57.756218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c713b96e-4f21-42b7-b33a-042452d0d6bb None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:57.757405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c713b96e-4f21-42b7-b33a-042452d0d6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:57.826064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 531/1047] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:58.096145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a0473-96e4-457a-b6b0-c8b215e1ebcb None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:58.140300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:58.204880 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a0473-96e4-457a-b6b0-c8b215e1ebcb None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:58.239607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fc227-2681-4a18-90d1-c3b9b578c5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:58.240260 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fc227-2681-4a18-90d1-c3b9b578c5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:58.240542 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fc227-2681-4a18-90d1-c3b9b578c5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:58.240815 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fc227-2681-4a18-90d1-c3b9b578c5b6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:58.241392 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067fc227-2681-4a18-90d1-c3b9b578c5b6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:58.242769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067fc227-2681-4a18-90d1-c3b9b578c5b6 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-067fc227-2681-4a18-90d1-c3b9b578c5b6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:58.252194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:58.253321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:58.253321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:58.253321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:58.253321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:58.253321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:58.253321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:58.253321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:20:58.253321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:20:58.253321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:20:58.253321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:20:58.253321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:58.253321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:58.253321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:58.253321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:58.253321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:58.253321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:58.254184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:58.254184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:58.254184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:58.254184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:58.254184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:58.254184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:58.254184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:58.254184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:58.254184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:58.254184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:58.254184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:20:58.254184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:20:58.254184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 12:20:58.254184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:20:58.255558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 517/1048] 10.222.0.58 () {56 vars in 1105 bytes} [Wed Aug 24 12:20:58 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:58.259826 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713b96e-4f21-42b7-b33a-042452d0d6bb None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:58.350985 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:58.357325 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa46769-cdc6-468c-8ee7-96ba96d498ec None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:58.357650 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa46769-cdc6-468c-8ee7-96ba96d498ec None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:58.357916 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa46769-cdc6-468c-8ee7-96ba96d498ec None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:58.359663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 518/1049] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:58.364205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7e3eb-1d03-4f0d-89ad-86c4ca8416d0 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:58.364557 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7e3eb-1d03-4f0d-89ad-86c4ca8416d0 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:58.364910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7e3eb-1d03-4f0d-89ad-86c4ca8416d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:58.365155 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713b96e-4f21-42b7-b33a-042452d0d6bb None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:58.383400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-cae7e3eb-1d03-4f0d-89ad-86c4ca8416d0 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:58.385546 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-cae7e3eb-1d03-4f0d-89ad-86c4ca8416d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:58.401900 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d35c6-187c-45f6-b3b7-4b15aa4e4e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:58.402510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d35c6-187c-45f6-b3b7-4b15aa4e4e09 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:58.402790 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d35c6-187c-45f6-b3b7-4b15aa4e4e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:58.403062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d35c6-187c-45f6-b3b7-4b15aa4e4e09 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:58.403720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d35c6-187c-45f6-b3b7-4b15aa4e4e09 None admin] RBAC: Adding query filter params (name=image) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:58.404026 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d35c6-187c-45f6-b3b7-4b15aa4e4e09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:58.405424 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d35c6-187c-45f6-b3b7-4b15aa4e4e09 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:58.411572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 532/1050] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:58.417599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d35c6-187c-45f6-b3b7-4b15aa4e4e09 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:58.461108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 519/1051] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:20:58.524221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d35c6-187c-45f6-b3b7-4b15aa4e4e09 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:58.558760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c023e-c270-46b7-bd5f-1c034aedba12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:58.559371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c023e-c270-46b7-bd5f-1c034aedba12 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:58.559670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c023e-c270-46b7-bd5f-1c034aedba12 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:58.559948 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c023e-c270-46b7-bd5f-1c034aedba12 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:58.560578 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c023e-c270-46b7-bd5f-1c034aedba12 None admin] RBAC: Adding query filter params (type=image) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:58.560882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c023e-c270-46b7-bd5f-1c034aedba12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:58.562251 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c023e-c270-46b7-bd5f-1c034aedba12 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:58.571888 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 533/1052] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:58.572968 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7e3eb-1d03-4f0d-89ad-86c4ca8416d0 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:58.604404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:58.673509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7e3eb-1d03-4f0d-89ad-86c4ca8416d0 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:58.708038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de7301-fe92-4cd9-b22e-895f263623a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:58.708642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de7301-fe92-4cd9-b22e-895f263623a9 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:58.708927 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de7301-fe92-4cd9-b22e-895f263623a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:58.709196 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de7301-fe92-4cd9-b22e-895f263623a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:58.709846 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de7301-fe92-4cd9-b22e-895f263623a9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:58.711215 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de7301-fe92-4cd9-b22e-895f263623a9 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:58.732242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 520/1053] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:20:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:20:58.744523 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c023e-c270-46b7-bd5f-1c034aedba12 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:58.769421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:58.859495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c023e-c270-46b7-bd5f-1c034aedba12 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:58.869569 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c1cbf-1740-4aaa-90f2-1d91fdfe90d4 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:58.869863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c1cbf-1740-4aaa-90f2-1d91fdfe90d4 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:58.870132 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c1cbf-1740-4aaa-90f2-1d91fdfe90d4 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:58.871658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 521/1054] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:20:58.877727 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f08ea-488e-4f31-922f-dfd9a50035e7 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:58.878009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f08ea-488e-4f31-922f-dfd9a50035e7 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:58.878291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f08ea-488e-4f31-922f-dfd9a50035e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:58.901266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-e04f08ea-488e-4f31-922f-dfd9a50035e7 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:20:58.901808 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e04f08ea-488e-4f31-922f-dfd9a50035e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:20:58.913198 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3dc330-75b3-4ac7-a228-fd0a483622cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:58.913828 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3dc330-75b3-4ac7-a228-fd0a483622cc None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:58.914108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3dc330-75b3-4ac7-a228-fd0a483622cc None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:58.914377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3dc330-75b3-4ac7-a228-fd0a483622cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:58.925136 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3dc330-75b3-4ac7-a228-fd0a483622cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:58.932117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-3b3dc330-75b3-4ac7-a228-fd0a483622cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:58.932117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:58.932117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:58.932117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:58.932117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:58.932117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:58.932117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:58.932117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:58.932117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:58.932117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:58.932117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:58.932117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:58.932117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:58.932117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:58.932117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:58.932117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:58.933093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:58.933093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:58.933093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:58.933093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:58.933093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:58.933093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:58.933093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:58.933093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:58.933093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:58.933093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:58.933093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:58.933093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:58.933093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:58.933093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:58.933093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:58.933093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:58.933982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:58.933982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:58.933982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:58.933982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:58.933982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:58.933982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:58.933982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:58.933982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:58.933982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:58.933982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:58.933982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:20:58.933982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:58.934675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 534/1055] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:20:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:58.939700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3dc330-75b3-4ac7-a228-fd0a483622cc None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:58.977976 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 522/1056] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:20:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:20:59.022622 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:59.046227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3dc330-75b3-4ac7-a228-fd0a483622cc None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:59.080972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa8b94-af96-4e7b-9c47-23cac802730f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:59.081579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa8b94-af96-4e7b-9c47-23cac802730f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.081855 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa8b94-af96-4e7b-9c47-23cac802730f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.082148 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa8b94-af96-4e7b-9c47-23cac802730f None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.082802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa8b94-af96-4e7b-9c47-23cac802730f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:59.083101 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa8b94-af96-4e7b-9c47-23cac802730f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:59.084490 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa8b94-af96-4e7b-9c47-23cac802730f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:59.097493 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 535/1057] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:20:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:59.098965 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f08ea-488e-4f31-922f-dfd9a50035e7 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:59.118714 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:59.202814 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f08ea-488e-4f31-922f-dfd9a50035e7 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:59.238387 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433af097-1099-47b4-bd49-fe4ae84e1bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:59.239023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433af097-1099-47b4-bd49-fe4ae84e1bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.239305 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433af097-1099-47b4-bd49-fe4ae84e1bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.239577 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433af097-1099-47b4-bd49-fe4ae84e1bf6 None admin] PATH_INFO: `/v3/users/634a89057c704eb4894648a096a1e9f0` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.252455 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa8b94-af96-4e7b-9c47-23cac802730f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:59.254154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433af097-1099-47b4-bd49-fe4ae84e1bf6 None admin] RBAC: Authorizing `identity:get_user(user_id=634a89057c704eb4894648a096a1e9f0)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:59.255610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433af097-1099-47b4-bd49-fe4ae84e1bf6 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:59.262634 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 523/1058] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:20:59 2022] GET /identity/v3/users/634a89057c704eb4894648a096a1e9f0 => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:20:59.263615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433af097-1099-47b4-bd49-fe4ae84e1bf6 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:59.343758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:59.360074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa8b94-af96-4e7b-9c47-23cac802730f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:59.370762 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433af097-1099-47b4-bd49-fe4ae84e1bf6 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:59.395579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da7d9e-f383-431a-a58b-c9a02c433b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:59.396344 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da7d9e-f383-431a-a58b-c9a02c433b8d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.396621 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da7d9e-f383-431a-a58b-c9a02c433b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.397006 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da7d9e-f383-431a-a58b-c9a02c433b8d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.404050 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:59.404602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cc6e3-e02c-4020-9296-7f655c346212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:59.405268 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cc6e3-e02c-4020-9296-7f655c346212 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.405538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cc6e3-e02c-4020-9296-7f655c346212 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.405781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cc6e3-e02c-4020-9296-7f655c346212 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.410571 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da7d9e-f383-431a-a58b-c9a02c433b8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:20:59.416206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d4da7d9e-f383-431a-a58b-c9a02c433b8d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:20:59.416206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:59.416206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:59.416206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:59.416206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:59.416206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:59.416206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:59.416206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:59.416206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:59.416206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:59.416206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:59.416206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:59.416206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:20:59.416206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:20:59.416206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:20:59.416206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:20:59.417163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:20:59.417163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:20:59.417163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:20:59.417163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:20:59.417163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:59.417163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:59.417163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:59.417163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:59.417163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:59.417163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:59.417163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:59.417163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:59.417163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:59.417163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:59.417163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:59.417163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:59.418037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:59.418037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:59.418037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:59.418037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:59.418037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:20:59.418037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:20:59.418037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:20:59.418037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:20:59.418037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:20:59.418037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:20:59.418037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:20:59.418037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:59.418778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 536/1059] 10.222.0.58 () {56 vars in 1120 bytes} [Wed Aug 24 12:20:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:20:59.424343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da7d9e-f383-431a-a58b-c9a02c433b8d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:59.483959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cc6e3-e02c-4020-9296-7f655c346212 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:59.485423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cc6e3-e02c-4020-9296-7f655c346212 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:59.525081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da7d9e-f383-431a-a58b-c9a02c433b8d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:59.559333 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efd3b1-0115-459c-a5ac-c0f5792b617f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:59.559952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efd3b1-0115-459c-a5ac-c0f5792b617f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.560230 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efd3b1-0115-459c-a5ac-c0f5792b617f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.560498 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efd3b1-0115-459c-a5ac-c0f5792b617f None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.561139 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efd3b1-0115-459c-a5ac-c0f5792b617f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:59.561434 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efd3b1-0115-459c-a5ac-c0f5792b617f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:59.562785 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efd3b1-0115-459c-a5ac-c0f5792b617f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:59.563843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:59.572037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 537/1060] 10.222.0.58 () {56 vars in 1115 bytes} [Wed Aug 24 12:20:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:59.577707 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efd3b1-0115-459c-a5ac-c0f5792b617f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:59.596150 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 524/1061] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:20:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:20:59.597153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cc6e3-e02c-4020-9296-7f655c346212 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:59.644484 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:59.687512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efd3b1-0115-459c-a5ac-c0f5792b617f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:59.707126 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cc6e3-e02c-4020-9296-7f655c346212 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:59.724242 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19439739-ca33-43fc-83a7-1bd58bfc6e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:59.725028 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19439739-ca33-43fc-83a7-1bd58bfc6e58 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.725311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19439739-ca33-43fc-83a7-1bd58bfc6e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.725582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19439739-ca33-43fc-83a7-1bd58bfc6e58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.738930 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19439739-ca33-43fc-83a7-1bd58bfc6e58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:59.740479 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19439739-ca33-43fc-83a7-1bd58bfc6e58 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:59.744374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30632d-3bb3-4beb-9eef-95b399bbf1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:59.745010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30632d-3bb3-4beb-9eef-95b399bbf1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.745291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30632d-3bb3-4beb-9eef-95b399bbf1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.745562 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30632d-3bb3-4beb-9eef-95b399bbf1c1 None admin] PATH_INFO: `/v3/projects/274a1dbd7041461a996def94f6dc0656` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.746670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30632d-3bb3-4beb-9eef-95b399bbf1c1 None admin] RBAC: Authorizing `identity:get_project(project_id=274a1dbd7041461a996def94f6dc0656)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:59.748122 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30632d-3bb3-4beb-9eef-95b399bbf1c1 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:59.754774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 525/1062] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:20:59 2022] GET /identity/v3/projects/274a1dbd7041461a996def94f6dc0656 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:59.755801 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30632d-3bb3-4beb-9eef-95b399bbf1c1 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:59.760532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-19439739-ca33-43fc-83a7-1bd58bfc6e58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:59.760532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:20:59.760532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:20:59.760532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:20:59.760532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:20:59.760532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:20:59.760532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:20:59.760532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:20:59.760532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:20:59.760532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:20:59.760532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:20:59.760532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:20:59.760532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:20:59.760532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:20:59.760532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:20:59.760532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:20:59.761802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:20:59.762730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:20:59.762730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:20:59.762730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:20:59.762730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 538/1063] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:20:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:20:59.767128 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19439739-ca33-43fc-83a7-1bd58bfc6e58 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:20:59.811928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:59.873355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30632d-3bb3-4beb-9eef-95b399bbf1c1 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:59.883608 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19439739-ca33-43fc-83a7-1bd58bfc6e58 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:20:59.897655 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:20:59.910801 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ed8ee-b706-4d50-adae-11dde31c3759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:59.911414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ed8ee-b706-4d50-adae-11dde31c3759 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.911729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ed8ee-b706-4d50-adae-11dde31c3759 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.912064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ed8ee-b706-4d50-adae-11dde31c3759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.912798 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ed8ee-b706-4d50-adae-11dde31c3759 None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=274a1dbd7041461a996def94f6dc0656, user.id=634a89057c704eb4894648a096a1e9f0) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:59.913106 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ed8ee-b706-4d50-adae-11dde31c3759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:59.914537 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8ed8ee-b706-4d50-adae-11dde31c3759 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:59.920661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 526/1064] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:20:59 2022] GET /identity/v3/role_assignments?user.id=634a89057c704eb4894648a096a1e9f0&scope.project.id=274a1dbd7041461a996def94f6dc0656&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:59.929637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a0dbf-85f0-433c-a2f6-1864ebb1c2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:20:59.930276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a0dbf-85f0-433c-a2f6-1864ebb1c2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:20:59.930593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a0dbf-85f0-433c-a2f6-1864ebb1c2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:20:59.930863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a0dbf-85f0-433c-a2f6-1864ebb1c2d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:20:59.931525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a0dbf-85f0-433c-a2f6-1864ebb1c2d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:20:59.931923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a0dbf-85f0-433c-a2f6-1864ebb1c2d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:20:59.933347 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a0dbf-85f0-433c-a2f6-1864ebb1c2d7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:20:59.946775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 539/1065] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:20:59.952697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ed8ee-b706-4d50-adae-11dde31c3759 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:00.075876 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ed8ee-b706-4d50-adae-11dde31c3759 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:00.085392 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:00.112359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f8efa-c53d-4efe-ad54-b16b5b7a082f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:00.112990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8efa-c53d-4efe-ad54-b16b5b7a082f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:00.113268 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8efa-c53d-4efe-ad54-b16b5b7a082f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:00.113537 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f8efa-c53d-4efe-ad54-b16b5b7a082f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:00.121450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f8efa-c53d-4efe-ad54-b16b5b7a082f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:00.122893 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f8efa-c53d-4efe-ad54-b16b5b7a082f None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:00.135774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-5c2f8efa-c53d-4efe-ad54-b16b5b7a082f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:21:00.135774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:00.135774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:00.135774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:00.135774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:00.135774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:00.135774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:00.135774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:00.135774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:00.135774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:00.135774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:00.135774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:00.135774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:21:00.135774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:21:00.135774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:21:00.135774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:00.136833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:00.137841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:00.137841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:00.137841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:21:00.137841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:21:00.137841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:21:00.137841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:21:00.137841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:21:00.137841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:21:00.137841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:21:00.137841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:21:00.139692 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 527/1066] 10.222.0.58 () {56 vars in 1093 bytes} [Wed Aug 24 12:20:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:00.142765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a0dbf-85f0-433c-a2f6-1864ebb1c2d7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:00.253121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a0dbf-85f0-433c-a2f6-1864ebb1c2d7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:00.288293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d477560-11fc-41db-98fb-853b570823b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:00.288905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d477560-11fc-41db-98fb-853b570823b1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:00.289186 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d477560-11fc-41db-98fb-853b570823b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:00.289458 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d477560-11fc-41db-98fb-853b570823b1 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:00.290486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d477560-11fc-41db-98fb-853b570823b1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:00.290799 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d477560-11fc-41db-98fb-853b570823b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:00.292264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d477560-11fc-41db-98fb-853b570823b1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:00.313896 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 540/1067] 10.222.0.58 () {56 vars in 1133 bytes} [Wed Aug 24 12:21:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:00.315379 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f8efa-c53d-4efe-ad54-b16b5b7a082f None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:00.425640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f8efa-c53d-4efe-ad54-b16b5b7a082f None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:00.460197 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38120d63-215d-4ede-a4f9-6e8e31f291d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:00.460831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38120d63-215d-4ede-a4f9-6e8e31f291d0 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:00.461114 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38120d63-215d-4ede-a4f9-6e8e31f291d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:00.461388 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38120d63-215d-4ede-a4f9-6e8e31f291d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:00.472810 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38120d63-215d-4ede-a4f9-6e8e31f291d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:00.474276 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38120d63-215d-4ede-a4f9-6e8e31f291d0 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:00.488439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-38120d63-215d-4ede-a4f9-6e8e31f291d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:00.488439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:00.488439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:00.488439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:00.488439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:00.488439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:00.488439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:00.488439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:00.488439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:00.488439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:00.488439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:00.488439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:00.488439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:21:00.488439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:21:00.488439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:21:00.488439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:00.489441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:21:00.490359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:21:00.490359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:00.490359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:21:00.491986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 528/1068] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:00.496119 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d477560-11fc-41db-98fb-853b570823b1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:00.528913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:00.601923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d477560-11fc-41db-98fb-853b570823b1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:00.636783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1eb71b-f12d-48f5-91ec-850c85413ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:00.637387 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1eb71b-f12d-48f5-91ec-850c85413ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:00.637670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1eb71b-f12d-48f5-91ec-850c85413ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:00.637947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1eb71b-f12d-48f5-91ec-850c85413ee3 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:00.638584 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1eb71b-f12d-48f5-91ec-850c85413ee3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:00.638890 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1eb71b-f12d-48f5-91ec-850c85413ee3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:00.640352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1eb71b-f12d-48f5-91ec-850c85413ee3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:00.656384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 541/1069] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:00.657989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38120d63-215d-4ede-a4f9-6e8e31f291d0 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:00.682666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:00.710955 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096699d5-7418-4992-8a60-058e3d65625d None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:00.711335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096699d5-7418-4992-8a60-058e3d65625d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:00.711506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096699d5-7418-4992-8a60-058e3d65625d None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:00.713418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 542/1070] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:21:00.718421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077bef5-6c5b-4fd1-b02b-f405071b2b73 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:00.718703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077bef5-6c5b-4fd1-b02b-f405071b2b73 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:00.719113 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077bef5-6c5b-4fd1-b02b-f405071b2b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:00.737615 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2077bef5-6c5b-4fd1-b02b-f405071b2b73 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:00.738791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2077bef5-6c5b-4fd1-b02b-f405071b2b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:00.789503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38120d63-215d-4ede-a4f9-6e8e31f291d0 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:00.812025 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 543/1071] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:21:00.832176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8b009-11f1-49fc-af59-b697d757fe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:00.833015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8b009-11f1-49fc-af59-b697d757fe7a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:00.833444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8b009-11f1-49fc-af59-b697d757fe7a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:00.833965 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8b009-11f1-49fc-af59-b697d757fe7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:00.840947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8b009-11f1-49fc-af59-b697d757fe7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:00.842562 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8b009-11f1-49fc-af59-b697d757fe7a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:00.854327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-a1b8b009-11f1-49fc-af59-b697d757fe7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:21:00.854327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:00.854327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:00.854327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:00.854327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:00.854327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:00.854327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:00.854327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:00.854327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:00.854327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:00.854327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:00.854327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:00.854327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:21:00.854327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:21:00.854327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:00.855384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:00.856535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:21:00.856535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:21:00.856535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:21:00.856535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:21:00.856535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:21:00.856535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:21:00.856535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:21:00.856535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:21:00.858494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 529/1072] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:21:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:21:00.864268 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077bef5-6c5b-4fd1-b02b-f405071b2b73 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:00.886541 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:00.968467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077bef5-6c5b-4fd1-b02b-f405071b2b73 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:01.004707 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f15b2d-a457-4fe1-a61a-7d1beb2e23bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:01.005625 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f15b2d-a457-4fe1-a61a-7d1beb2e23bb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:01.006054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f15b2d-a457-4fe1-a61a-7d1beb2e23bb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:01.006476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f15b2d-a457-4fe1-a61a-7d1beb2e23bb None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:01.007339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f15b2d-a457-4fe1-a61a-7d1beb2e23bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:01.007863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f15b2d-a457-4fe1-a61a-7d1beb2e23bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:01.009524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f15b2d-a457-4fe1-a61a-7d1beb2e23bb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:01.021905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 544/1073] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:21:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:01.028291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8b009-11f1-49fc-af59-b697d757fe7a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:01.086680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f15b2d-a457-4fe1-a61a-7d1beb2e23bb None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:01.112360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:01.134208 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8b009-11f1-49fc-af59-b697d757fe7a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:01.169986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29668531-8585-4658-830c-0fe734aea3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:01.170635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29668531-8585-4658-830c-0fe734aea3c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:01.170911 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29668531-8585-4658-830c-0fe734aea3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:01.171183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29668531-8585-4658-830c-0fe734aea3c2 None admin] PATH_INFO: `/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/311dbed3ab0b44a7830a93227ff9ad33/roles/4c5d21de669d42169b9218cb54f295e4` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:01.173284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29668531-8585-4658-830c-0fe734aea3c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user_id=311dbed3ab0b44a7830a93227ff9ad33, role_id=4c5d21de669d42169b9218cb54f295e4)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:01.174117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:01.174708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29668531-8585-4658-830c-0fe734aea3c2 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:01.188945 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 530/1074] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:21:01 2022] PUT /identity/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/311dbed3ab0b44a7830a93227ff9ad33/roles/4c5d21de669d42169b9218cb54f295e4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:21:01.195559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f15b2d-a457-4fe1-a61a-7d1beb2e23bb None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:01.234543 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f0753-864d-4f9e-8655-9cf6adeeb397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:01.235192 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f0753-864d-4f9e-8655-9cf6adeeb397 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:01.235458 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f0753-864d-4f9e-8655-9cf6adeeb397 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:01.235790 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f0753-864d-4f9e-8655-9cf6adeeb397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:01.310257 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f0753-864d-4f9e-8655-9cf6adeeb397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:01.311769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f0753-864d-4f9e-8655-9cf6adeeb397 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:01.405190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:01.427147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 545/1075] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 12:21:01.433247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29668531-8585-4658-830c-0fe734aea3c2 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:01.536606 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29668531-8585-4658-830c-0fe734aea3c2 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:01.570468 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813924f1-24a7-409b-a6ac-8eeb11c932e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:01.571652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813924f1-24a7-409b-a6ac-8eeb11c932e1 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:01.571950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813924f1-24a7-409b-a6ac-8eeb11c932e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:01.572248 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813924f1-24a7-409b-a6ac-8eeb11c932e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:01.581180 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:01.584194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813924f1-24a7-409b-a6ac-8eeb11c932e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:01.586207 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813924f1-24a7-409b-a6ac-8eeb11c932e1 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:01.603409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-813924f1-24a7-409b-a6ac-8eeb11c932e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:01.603409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:01.603409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:01.603409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:01.603409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:01.603409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:01.603409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:01.603409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:01.603409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:01.603409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:01.603409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:01.603409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:01.603409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:21:01.603409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:21:01.603409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:21:01.603409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:01.604649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:21:01.605958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:21:01.605958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:01.605958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:21:01.607686 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 531/1076] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:01.610528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f0753-864d-4f9e-8655-9cf6adeeb397 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:01.714414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f0753-864d-4f9e-8655-9cf6adeeb397 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:01.749753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836bb93-aa9d-41e2-8b54-7c199d9ba4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:01.750362 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836bb93-aa9d-41e2-8b54-7c199d9ba4e1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:01.750645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836bb93-aa9d-41e2-8b54-7c199d9ba4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:01.750922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836bb93-aa9d-41e2-8b54-7c199d9ba4e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:01.751558 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836bb93-aa9d-41e2-8b54-7c199d9ba4e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:01.751891 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836bb93-aa9d-41e2-8b54-7c199d9ba4e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:01.753266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836bb93-aa9d-41e2-8b54-7c199d9ba4e1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:01.771661 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 546/1077] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:01.845315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a548e74-d50a-4a4b-914a-1a11a38489ed None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:01.845905 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a548e74-d50a-4a4b-914a-1a11a38489ed None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:01.845905 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a548e74-d50a-4a4b-914a-1a11a38489ed None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:01.847475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 532/1078] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:01.853884 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c67cd-c64d-41d4-9abc-892abef55c7e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:01.854375 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c67cd-c64d-41d4-9abc-892abef55c7e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:01.854448 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c67cd-c64d-41d4-9abc-892abef55c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:01.875665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-081c67cd-c64d-41d4-9abc-892abef55c7e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:01.875665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-081c67cd-c64d-41d4-9abc-892abef55c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:01.918523 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:01.953192 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 547/1079] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:02.232703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a548e74-d50a-4a4b-914a-1a11a38489ed None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:02.336945 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a548e74-d50a-4a4b-914a-1a11a38489ed None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:02.370899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946b55f-2593-4195-b09d-e3b7ca42b283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:02.371525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946b55f-2593-4195-b09d-e3b7ca42b283 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:02.371835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946b55f-2593-4195-b09d-e3b7ca42b283 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:02.372112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946b55f-2593-4195-b09d-e3b7ca42b283 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:02.382038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946b55f-2593-4195-b09d-e3b7ca42b283 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:02.386924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-e946b55f-2593-4195-b09d-e3b7ca42b283 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:02.386924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:02.386924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:02.386924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:02.386924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:02.386924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:02.386924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:02.386924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:02.386924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:02.386924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:02.386924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:02.386924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:02.386924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:02.386924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:02.386924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:02.386924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:02.388091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:02.388091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:02.388091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:02.388091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:02.388091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:02.388091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:02.388091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:02.388091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:02.388091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:02.388091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:02.388091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:02.388091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:02.388091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:02.388091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:02.388091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:02.388091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:02.389200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:02.389200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:02.389200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:02.389200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:02.389200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:02.389200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:02.389200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:02.389200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:02.389200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:02.389200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:02.389200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:02.389200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:21:02.391775 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 533/1080] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:21:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:02.393955 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c67cd-c64d-41d4-9abc-892abef55c7e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:02.498356 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c67cd-c64d-41d4-9abc-892abef55c7e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:02.530854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae75a6-2edd-4080-85a9-96446faad0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:02.531458 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae75a6-2edd-4080-85a9-96446faad0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:02.531769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae75a6-2edd-4080-85a9-96446faad0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:02.532046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae75a6-2edd-4080-85a9-96446faad0b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:02.532702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae75a6-2edd-4080-85a9-96446faad0b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:02.533007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae75a6-2edd-4080-85a9-96446faad0b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:02.534574 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae75a6-2edd-4080-85a9-96446faad0b2 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:02.545197 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 548/1081] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:21:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:02.550921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946b55f-2593-4195-b09d-e3b7ca42b283 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:02.604534 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:02.652602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946b55f-2593-4195-b09d-e3b7ca42b283 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:02.685559 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ec497-78b7-448c-af52-c5431727dd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:02.686186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ec497-78b7-448c-af52-c5431727dd02 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:02.686467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ec497-78b7-448c-af52-c5431727dd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:02.686740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ec497-78b7-448c-af52-c5431727dd02 None admin] PATH_INFO: `/v3/users/634a89057c704eb4894648a096a1e9f0` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:02.687879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ec497-78b7-448c-af52-c5431727dd02 None admin] RBAC: Authorizing `identity:get_user(user_id=634a89057c704eb4894648a096a1e9f0)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:02.689344 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ec497-78b7-448c-af52-c5431727dd02 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:02.690937 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 534/1082] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:02 2022] GET /identity/v3/users/634a89057c704eb4894648a096a1e9f0 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:02.696712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae75a6-2edd-4080-85a9-96446faad0b2 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:02.729397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:02.797449 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae75a6-2edd-4080-85a9-96446faad0b2 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:02.830256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1faa5-3afe-4dd0-b259-879f886aa3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:02.831144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1faa5-3afe-4dd0-b259-879f886aa3a3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:02.831431 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1faa5-3afe-4dd0-b259-879f886aa3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:02.831732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1faa5-3afe-4dd0-b259-879f886aa3a3 None admin] PATH_INFO: `/v3/projects/274a1dbd7041461a996def94f6dc0656` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:02.832819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1faa5-3afe-4dd0-b259-879f886aa3a3 None admin] RBAC: Authorizing `identity:get_project(project_id=274a1dbd7041461a996def94f6dc0656)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:02.834232 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1faa5-3afe-4dd0-b259-879f886aa3a3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:02.835905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 549/1083] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:02 2022] GET /identity/v3/projects/274a1dbd7041461a996def94f6dc0656 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:02.842298 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ec497-78b7-448c-af52-c5431727dd02 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:02.940414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ec497-78b7-448c-af52-c5431727dd02 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:02.959393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:02.974291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0c833-34c1-4d0c-8b1f-c994d62c69ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:02.974945 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0c833-34c1-4d0c-8b1f-c994d62c69ff None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:02.975220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0c833-34c1-4d0c-8b1f-c994d62c69ff None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:02.975487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0c833-34c1-4d0c-8b1f-c994d62c69ff None admin] PATH_INFO: `/v3/projects/274a1dbd7041461a996def94f6dc0656/users/634a89057c704eb4894648a096a1e9f0/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:02.977976 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0c833-34c1-4d0c-8b1f-c994d62c69ff None admin] RBAC: Authorizing `identity:create_grant(project_id=274a1dbd7041461a996def94f6dc0656, user_id=634a89057c704eb4894648a096a1e9f0, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:02.979513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0c833-34c1-4d0c-8b1f-c994d62c69ff None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:02.992063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 535/1084] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:21:02 2022] PUT /identity/v3/projects/274a1dbd7041461a996def94f6dc0656/users/634a89057c704eb4894648a096a1e9f0/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:21:03.122576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793ed76-2368-4330-9be3-8f9e9a383ba9 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:03.123125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793ed76-2368-4330-9be3-8f9e9a383ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:03.123125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793ed76-2368-4330-9be3-8f9e9a383ba9 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:03.124793 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 550/1085] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:03.130395 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ba8da-4b28-46e5-ad23-ebd0d4f1344e None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:03.130679 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ba8da-4b28-46e5-ad23-ebd0d4f1344e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:03.130952 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ba8da-4b28-46e5-ad23-ebd0d4f1344e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:03.150608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-109ba8da-4b28-46e5-ad23-ebd0d4f1344e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:03.151821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-109ba8da-4b28-46e5-ad23-ebd0d4f1344e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:03.217727 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 536/1086] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:03.486939 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793ed76-2368-4330-9be3-8f9e9a383ba9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:03.583014 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:03.587755 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793ed76-2368-4330-9be3-8f9e9a383ba9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:03.620585 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46526d55-c087-4204-bf4f-300afaddc633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:03.621209 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46526d55-c087-4204-bf4f-300afaddc633 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:03.621489 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46526d55-c087-4204-bf4f-300afaddc633 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:03.621761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46526d55-c087-4204-bf4f-300afaddc633 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:03.632011 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46526d55-c087-4204-bf4f-300afaddc633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:03.637426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-46526d55-c087-4204-bf4f-300afaddc633 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:21:03.637426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:03.637426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:03.637426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:03.637426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:03.637426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:03.637426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:03.637426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:03.637426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:03.637426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:03.637426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:03.637426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:03.637426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:03.637426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:03.637426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:03.637426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:03.638379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:03.638379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:03.638379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:03.638379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:03.638379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:03.638379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:03.638379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:03.638379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:03.638379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:03.638379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:03.638379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:03.638379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:03.638379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:03.638379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:03.638379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:03.638379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:03.639259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:03.639259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:03.639259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:03.639259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:03.639259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:03.639259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:03.639259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:03.639259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:03.639259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:03.639259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:03.639259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 12:21:03.639259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:21:03.639259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 551/1087] 10.222.0.58 () {56 vars in 1120 bytes} [Wed Aug 24 12:21:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:03.644489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ba8da-4b28-46e5-ad23-ebd0d4f1344e None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:03.745480 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ba8da-4b28-46e5-ad23-ebd0d4f1344e None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:03.755710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:03.780754 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ebbfb-904e-40ca-85f3-7523428b27c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:03.781361 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ebbfb-904e-40ca-85f3-7523428b27c3 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:03.781642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ebbfb-904e-40ca-85f3-7523428b27c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:03.781915 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ebbfb-904e-40ca-85f3-7523428b27c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:03.782572 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ebbfb-904e-40ca-85f3-7523428b27c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:03.782875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ebbfb-904e-40ca-85f3-7523428b27c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:03.784273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ebbfb-904e-40ca-85f3-7523428b27c3 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:03.792726 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 537/1088] 10.222.0.58 () {56 vars in 1115 bytes} [Wed Aug 24 12:21:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:03.798449 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46526d55-c087-4204-bf4f-300afaddc633 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:03.902314 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46526d55-c087-4204-bf4f-300afaddc633 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:03.936017 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e64a92-f079-4ac7-9f77-caa8cf32f657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:03.936644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e64a92-f079-4ac7-9f77-caa8cf32f657 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:03.936925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e64a92-f079-4ac7-9f77-caa8cf32f657 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:03.937200 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e64a92-f079-4ac7-9f77-caa8cf32f657 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:03.947813 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e64a92-f079-4ac7-9f77-caa8cf32f657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:03.949262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e64a92-f079-4ac7-9f77-caa8cf32f657 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:03.963397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-66e64a92-f079-4ac7-9f77-caa8cf32f657 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:03.963397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:03.963397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:03.963397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:03.963397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:03.963397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:03.963397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:03.963397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:03.963397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:03.963397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:03.963397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:03.963397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:03.963397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:21:03.963397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:21:03.963397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:21:03.963397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:03.964612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:21:03.965517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:21:03.965517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:03.965517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:21:03.965517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 552/1089] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:21:03.970395 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ebbfb-904e-40ca-85f3-7523428b27c3 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:04.073924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ebbfb-904e-40ca-85f3-7523428b27c3 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:04.107237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7efe72-baea-4d7a-bab9-d926588ccb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:04.107865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7efe72-baea-4d7a-bab9-d926588ccb96 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:04.108150 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7efe72-baea-4d7a-bab9-d926588ccb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:04.108422 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7efe72-baea-4d7a-bab9-d926588ccb96 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:04.109061 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7efe72-baea-4d7a-bab9-d926588ccb96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:04.109368 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7efe72-baea-4d7a-bab9-d926588ccb96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:04.110731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7efe72-baea-4d7a-bab9-d926588ccb96 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:04.122606 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 538/1090] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:04.128240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e64a92-f079-4ac7-9f77-caa8cf32f657 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:04.143533 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:04.228822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e64a92-f079-4ac7-9f77-caa8cf32f657 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:04.260991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7beda7-d413-4e84-817e-08c73d8b9798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:04.261614 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7beda7-d413-4e84-817e-08c73d8b9798 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:04.261895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7beda7-d413-4e84-817e-08c73d8b9798 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:04.262171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7beda7-d413-4e84-817e-08c73d8b9798 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:04.269390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7beda7-d413-4e84-817e-08c73d8b9798 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:04.270822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7beda7-d413-4e84-817e-08c73d8b9798 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:04.282121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-8a7beda7-d413-4e84-817e-08c73d8b9798 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:21:04.282121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:04.282121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:04.282121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:04.282121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:04.282121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:04.282121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:04.282121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:04.282121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:04.282121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:04.282121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:04.282121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:04.282121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:21:04.282121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:21:04.282121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:21:04.282121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:04.283166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:04.284170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:04.284170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:04.284170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:21:04.284170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:21:04.284170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:21:04.284170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:21:04.284170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:21:04.284170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:21:04.284170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 12:21:04.284170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:21:04.284170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 553/1091] 10.222.0.58 () {56 vars in 1093 bytes} [Wed Aug 24 12:21:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:04.289025 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7efe72-baea-4d7a-bab9-d926588ccb96 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:04.320064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:04.388224 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7efe72-baea-4d7a-bab9-d926588ccb96 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:04.422776 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efd7ca-9c0a-4dc1-ab57-7776d3d1351f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:04.423441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efd7ca-9c0a-4dc1-ab57-7776d3d1351f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:04.423745 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efd7ca-9c0a-4dc1-ab57-7776d3d1351f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:04.424013 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efd7ca-9c0a-4dc1-ab57-7776d3d1351f None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:04.424731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efd7ca-9c0a-4dc1-ab57-7776d3d1351f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:04.425031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efd7ca-9c0a-4dc1-ab57-7776d3d1351f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:04.426438 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efd7ca-9c0a-4dc1-ab57-7776d3d1351f None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:04.443660 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 539/1092] 10.222.0.58 () {56 vars in 1133 bytes} [Wed Aug 24 12:21:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:04.449307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7beda7-d413-4e84-817e-08c73d8b9798 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:04.518746 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:04.549814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7beda7-d413-4e84-817e-08c73d8b9798 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:04.583408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d7917-da44-4f93-9579-8ec55388f528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:04.584033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d7917-da44-4f93-9579-8ec55388f528 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:04.584309 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d7917-da44-4f93-9579-8ec55388f528 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:04.584578 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d7917-da44-4f93-9579-8ec55388f528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:04.651855 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d7917-da44-4f93-9579-8ec55388f528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:04.653226 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d7917-da44-4f93-9579-8ec55388f528 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:04.719906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:04.750064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 554/1093] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:04.755944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efd7ca-9c0a-4dc1-ab57-7776d3d1351f None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:04.845288 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:04.855036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efd7ca-9c0a-4dc1-ab57-7776d3d1351f None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:04.892073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719f20f-d1fd-4cab-abc7-b0e460a09784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:04.892704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719f20f-d1fd-4cab-abc7-b0e460a09784 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:04.892983 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719f20f-d1fd-4cab-abc7-b0e460a09784 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:04.893254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719f20f-d1fd-4cab-abc7-b0e460a09784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:04.904039 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719f20f-d1fd-4cab-abc7-b0e460a09784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:04.905418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719f20f-d1fd-4cab-abc7-b0e460a09784 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:04.916930 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6b4bd-c764-48bf-9c21-54e83312ddbe None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:04.917219 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6b4bd-c764-48bf-9c21-54e83312ddbe None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:04.917499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6b4bd-c764-48bf-9c21-54e83312ddbe None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:04.918981 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 555/1094] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:04.920058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-7719f20f-d1fd-4cab-abc7-b0e460a09784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:04.920058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:04.920058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:04.920058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:04.920058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:04.920058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:04.920058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:04.920058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:04.920058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:04.920058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:04.920058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:04.920058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:04.920058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:21:04.920058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:21:04.920058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:21:04.920058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:04.921186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:21:04.922139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:21:04.922139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:21:04.922139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:21:04.922139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 540/1095] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:04.925201 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e95db7-0fa7-4ed2-a5f9-8ac849a75898 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:04.925484 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e95db7-0fa7-4ed2-a5f9-8ac849a75898 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:04.925761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e95db7-0fa7-4ed2-a5f9-8ac849a75898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:04.926407 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719f20f-d1fd-4cab-abc7-b0e460a09784 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:04.943812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e1e95db7-0fa7-4ed2-a5f9-8ac849a75898 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:04.945007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e1e95db7-0fa7-4ed2-a5f9-8ac849a75898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:05.012432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 556/1096] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:05.025390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719f20f-d1fd-4cab-abc7-b0e460a09784 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:05.058235 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b49201-8e90-4fbb-8cab-7f024c093861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:05.058834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b49201-8e90-4fbb-8cab-7f024c093861 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:05.059117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b49201-8e90-4fbb-8cab-7f024c093861 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:05.059393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b49201-8e90-4fbb-8cab-7f024c093861 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:05.060059 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b49201-8e90-4fbb-8cab-7f024c093861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:05.060366 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b49201-8e90-4fbb-8cab-7f024c093861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:05.061738 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b49201-8e90-4fbb-8cab-7f024c093861 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:05.073852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 541/1097] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:05.079588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e95db7-0fa7-4ed2-a5f9-8ac849a75898 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:05.176623 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e95db7-0fa7-4ed2-a5f9-8ac849a75898 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:05.178081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:05.213703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35249939-da4b-43d0-8caa-3dd882e18623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:05.214330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35249939-da4b-43d0-8caa-3dd882e18623 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:05.214613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35249939-da4b-43d0-8caa-3dd882e18623 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:05.214895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35249939-da4b-43d0-8caa-3dd882e18623 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:05.221449 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35249939-da4b-43d0-8caa-3dd882e18623 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:05.222889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35249939-da4b-43d0-8caa-3dd882e18623 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:05.234049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-35249939-da4b-43d0-8caa-3dd882e18623 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:21:05.234049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:05.234049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:05.234049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:05.234049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:05.234049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:05.234049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:05.234049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:05.234049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:05.234049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:05.234049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:05.234049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:05.234049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:21:05.234049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:21:05.234049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:05.235100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:05.236081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:21:05.236081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:21:05.236081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:21:05.236081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:21:05.236081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:21:05.236081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:21:05.236081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:21:05.236081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:21:05.236081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 557/1098] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:21:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:05.240945 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b49201-8e90-4fbb-8cab-7f024c093861 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:05.244337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:05.280827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35249939-da4b-43d0-8caa-3dd882e18623 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:05.351562 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b49201-8e90-4fbb-8cab-7f024c093861 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:05.387572 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7b240-32de-4361-a5f5-89ad624ec13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:05.388205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7b240-32de-4361-a5f5-89ad624ec13d None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:05.388489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7b240-32de-4361-a5f5-89ad624ec13d None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:05.388763 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7b240-32de-4361-a5f5-89ad624ec13d None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:05.389058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35249939-da4b-43d0-8caa-3dd882e18623 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:05.389456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b7b240-32de-4361-a5f5-89ad624ec13d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:05.389765 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b7b240-32de-4361-a5f5-89ad624ec13d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:05.391225 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b7b240-32de-4361-a5f5-89ad624ec13d None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:05.402874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 542/1099] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:21:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:05.408818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7b240-32de-4361-a5f5-89ad624ec13d None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:05.423606 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161407d-f484-441c-a660-d9ca3ba4e6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:05.424262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161407d-f484-441c-a660-d9ca3ba4e6fb None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:05.424542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161407d-f484-441c-a660-d9ca3ba4e6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:05.424817 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161407d-f484-441c-a660-d9ca3ba4e6fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:05.436347 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3161407d-f484-441c-a660-d9ca3ba4e6fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:05.441043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-3161407d-f484-441c-a660-d9ca3ba4e6fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:05.441043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:05.441043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:05.441043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:05.441043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:05.441043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:05.441043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:05.441043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:05.441043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:05.441043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:05.441043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:05.441043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:05.441043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:05.441043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:05.441043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:05.441043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:05.442000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:05.442000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:05.442000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:05.442000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:05.442000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:05.442000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:05.442000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:05.442000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:05.442000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:05.442000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:05.442000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:05.442000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:05.442000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:05.442000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:05.442000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:05.442000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:05.442871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:05.442871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:05.442871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:05.442871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:05.442871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:05.442871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:05.442871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:05.442871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:05.442871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:05.442871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:05.442871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:05.442871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:21:05.442871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 558/1100] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:21:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:21:05.448051 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161407d-f484-441c-a660-d9ca3ba4e6fb None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:05.470805 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:05.475901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:05.510328 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7b240-32de-4361-a5f5-89ad624ec13d None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:05.546765 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc7ca3-1c3d-4a6e-9b5f-8e7356d02765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:05.547369 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc7ca3-1c3d-4a6e-9b5f-8e7356d02765 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:05.547678 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc7ca3-1c3d-4a6e-9b5f-8e7356d02765 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:05.547953 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc7ca3-1c3d-4a6e-9b5f-8e7356d02765 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:05.548675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc7ca3-1c3d-4a6e-9b5f-8e7356d02765 None admin] RBAC: Adding query filter params (role.id=4c5d21de669d42169b9218cb54f295e4, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=311dbed3ab0b44a7830a93227ff9ad33) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:05.548982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc7ca3-1c3d-4a6e-9b5f-8e7356d02765 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:05.550411 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc7ca3-1c3d-4a6e-9b5f-8e7356d02765 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:05.556074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161407d-f484-441c-a660-d9ca3ba4e6fb None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:05.556826 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 543/1101] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:05 2022] GET /identity/v3/role_assignments?user.id=311dbed3ab0b44a7830a93227ff9ad33&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=4c5d21de669d42169b9218cb54f295e4 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:05.591584 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b522a-b53f-4d52-9b2b-77cfa4570006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:05.592246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b522a-b53f-4d52-9b2b-77cfa4570006 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:05.592529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b522a-b53f-4d52-9b2b-77cfa4570006 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:05.592801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b522a-b53f-4d52-9b2b-77cfa4570006 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:05.593476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b522a-b53f-4d52-9b2b-77cfa4570006 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:05.593782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b522a-b53f-4d52-9b2b-77cfa4570006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:05.595168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b522a-b53f-4d52-9b2b-77cfa4570006 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:05.610505 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 559/1102] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:21:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:05.611790 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc7ca3-1c3d-4a6e-9b5f-8e7356d02765 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:05.718060 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc7ca3-1c3d-4a6e-9b5f-8e7356d02765 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:05.744413 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:05.754274 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70119350-a4c3-4ad0-ade3-81aa23527eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:05.754911 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70119350-a4c3-4ad0-ade3-81aa23527eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:05.755192 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70119350-a4c3-4ad0-ade3-81aa23527eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:05.755466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70119350-a4c3-4ad0-ade3-81aa23527eb1 None admin] PATH_INFO: `/v3/users/634a89057c704eb4894648a096a1e9f0` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:05.756637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70119350-a4c3-4ad0-ade3-81aa23527eb1 None admin] RBAC: Authorizing `identity:get_user(user_id=634a89057c704eb4894648a096a1e9f0)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:05.758078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70119350-a4c3-4ad0-ade3-81aa23527eb1 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:05.760044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 544/1103] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:05 2022] GET /identity/v3/users/634a89057c704eb4894648a096a1e9f0 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:05.766109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b522a-b53f-4d52-9b2b-77cfa4570006 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:05.854811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:05.871271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b522a-b53f-4d52-9b2b-77cfa4570006 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:05.906319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba4fa6-cba2-430e-b5fd-b07fcbf90183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:05.906926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba4fa6-cba2-430e-b5fd-b07fcbf90183 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:05.907206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba4fa6-cba2-430e-b5fd-b07fcbf90183 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:05.907502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba4fa6-cba2-430e-b5fd-b07fcbf90183 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:05.977987 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecba4fa6-cba2-430e-b5fd-b07fcbf90183 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:05.979430 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecba4fa6-cba2-430e-b5fd-b07fcbf90183 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:06.062581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:06.080922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 560/1104] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:06.087398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70119350-a4c3-4ad0-ade3-81aa23527eb1 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:06.196331 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70119350-a4c3-4ad0-ade3-81aa23527eb1 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:06.230878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45159fb-5584-474f-b08c-101ed6635ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:06.231432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45159fb-5584-474f-b08c-101ed6635ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:06.231733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45159fb-5584-474f-b08c-101ed6635ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:06.232003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45159fb-5584-474f-b08c-101ed6635ce4 None admin] PATH_INFO: `/v3/projects/274a1dbd7041461a996def94f6dc0656` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:06.233222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45159fb-5584-474f-b08c-101ed6635ce4 None admin] RBAC: Authorizing `identity:get_project(project_id=274a1dbd7041461a996def94f6dc0656)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:06.234691 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45159fb-5584-474f-b08c-101ed6635ce4 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:06.236381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 545/1105] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:06 2022] GET /identity/v3/projects/274a1dbd7041461a996def94f6dc0656 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:06.242606 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba4fa6-cba2-430e-b5fd-b07fcbf90183 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:06.343919 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba4fa6-cba2-430e-b5fd-b07fcbf90183 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:06.380065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2bf787-a0da-4e32-91a1-3e734654a5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:06.380671 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bf787-a0da-4e32-91a1-3e734654a5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:06.380951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bf787-a0da-4e32-91a1-3e734654a5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:06.381225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bf787-a0da-4e32-91a1-3e734654a5e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:06.381953 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2bf787-a0da-4e32-91a1-3e734654a5e4 None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=274a1dbd7041461a996def94f6dc0656, user.id=634a89057c704eb4894648a096a1e9f0) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:06.382260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2bf787-a0da-4e32-91a1-3e734654a5e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:06.383848 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2bf787-a0da-4e32-91a1-3e734654a5e4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:06.389741 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 561/1106] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:06 2022] GET /identity/v3/role_assignments?user.id=634a89057c704eb4894648a096a1e9f0&scope.project.id=274a1dbd7041461a996def94f6dc0656&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:08.333134 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c267a8-9781-4910-8234-28e0a1fa7a8f None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:08.333658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c267a8-9781-4910-8234-28e0a1fa7a8f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:08.333658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c267a8-9781-4910-8234-28e0a1fa7a8f None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:08.335062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 546/1107] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:08.340956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b59480-ab01-4c96-9493-e023bc6ab91e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:08.341337 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b59480-ab01-4c96-9493-e023bc6ab91e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:08.341512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b59480-ab01-4c96-9493-e023bc6ab91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:08.359321 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d6b59480-ab01-4c96-9493-e023bc6ab91e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:08.360620 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d6b59480-ab01-4c96-9493-e023bc6ab91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:08.403415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:08.426059 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 562/1108] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:08.697535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c267a8-9781-4910-8234-28e0a1fa7a8f None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:08.760338 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:08.799291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c267a8-9781-4910-8234-28e0a1fa7a8f None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:08.832734 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888256aa-57e2-4ea0-8060-3b7b09eef6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:08.833352 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888256aa-57e2-4ea0-8060-3b7b09eef6dd None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:08.833630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888256aa-57e2-4ea0-8060-3b7b09eef6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:08.833900 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888256aa-57e2-4ea0-8060-3b7b09eef6dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:08.834980 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888256aa-57e2-4ea0-8060-3b7b09eef6dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:08.836459 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888256aa-57e2-4ea0-8060-3b7b09eef6dd None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:08.838015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 547/1109] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:08.843860 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b59480-ab01-4c96-9493-e023bc6ab91e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:08.942848 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b59480-ab01-4c96-9493-e023bc6ab91e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:08.976062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c9a28-fc44-4630-a98e-76470ae9440f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:08.976597 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9a28-fc44-4630-a98e-76470ae9440f None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:08.976902 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9a28-fc44-4630-a98e-76470ae9440f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:08.977221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9a28-fc44-4630-a98e-76470ae9440f None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:08.977836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c9a28-fc44-4630-a98e-76470ae9440f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:08.979260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c9a28-fc44-4630-a98e-76470ae9440f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:08.999724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 563/1110] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:21:08 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:21:10.953457 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a2b9d-39e0-4712-aae5-c016a244de4c None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:10.953961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a2b9d-39e0-4712-aae5-c016a244de4c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:10.954072 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a2b9d-39e0-4712-aae5-c016a244de4c None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:10.955334 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 548/1111] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:10.960502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480f20d-c4c7-4efa-a173-08f152394da2 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:10.960951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480f20d-c4c7-4efa-a173-08f152394da2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:10.961044 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480f20d-c4c7-4efa-a173-08f152394da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:10.978843 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e480f20d-c4c7-4efa-a173-08f152394da2 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:10.980075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e480f20d-c4c7-4efa-a173-08f152394da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:11.006425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:11.047766 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 564/1112] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:11.316914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a2b9d-39e0-4712-aae5-c016a244de4c None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:11.413666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a2b9d-39e0-4712-aae5-c016a244de4c None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:11.441486 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:11.447723 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8c2f5-d830-4bc0-bd96-c6bc605c938d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:11.448338 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8c2f5-d830-4bc0-bd96-c6bc605c938d None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:11.448616 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8c2f5-d830-4bc0-bd96-c6bc605c938d None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:11.448887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8c2f5-d830-4bc0-bd96-c6bc605c938d None admin] PATH_INFO: `/v3/roles/4e002a22056744acb8c40a3236cab404` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:11.450504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8c2f5-d830-4bc0-bd96-c6bc605c938d None admin] RBAC: Authorizing `identity:get_role(role_id=4e002a22056744acb8c40a3236cab404)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:11.451891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8c2f5-d830-4bc0-bd96-c6bc605c938d None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:11.453190 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 549/1113] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:11 2022] GET /identity/v3/roles/4e002a22056744acb8c40a3236cab404 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:11.458901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480f20d-c4c7-4efa-a173-08f152394da2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:11.553947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480f20d-c4c7-4efa-a173-08f152394da2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:11.587421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c6f3c-fb0d-49b9-b2b8-f7903309e8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:11.588023 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c6f3c-fb0d-49b9-b2b8-f7903309e8ca None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:11.588306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c6f3c-fb0d-49b9-b2b8-f7903309e8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:11.588580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c6f3c-fb0d-49b9-b2b8-f7903309e8ca None admin] PATH_INFO: `/v3/users/19f91bbfbfb24f6d8492ae1553e6ba0d` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:11.601206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c6f3c-fb0d-49b9-b2b8-f7903309e8ca None admin] RBAC: Authorizing `identity:get_user(user_id=19f91bbfbfb24f6d8492ae1553e6ba0d)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:11.603170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c6f3c-fb0d-49b9-b2b8-f7903309e8ca None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:11.604795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 565/1114] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:11 2022] GET /identity/v3/users/19f91bbfbfb24f6d8492ae1553e6ba0d => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:11.610400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8c2f5-d830-4bc0-bd96-c6bc605c938d None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:11.705482 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8c2f5-d830-4bc0-bd96-c6bc605c938d None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:11.738590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89fc9b-5a5b-49ac-9e18-c0fa5885175d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:11.739169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89fc9b-5a5b-49ac-9e18-c0fa5885175d None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:11.739447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89fc9b-5a5b-49ac-9e18-c0fa5885175d None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:11.739743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e89fc9b-5a5b-49ac-9e18-c0fa5885175d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:11.801270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:11.804379 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89fc9b-5a5b-49ac-9e18-c0fa5885175d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:11.806882 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e89fc9b-5a5b-49ac-9e18-c0fa5885175d None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:11.903607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 550/1115] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:11.909652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c6f3c-fb0d-49b9-b2b8-f7903309e8ca None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:11.934043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:12.007958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c6f3c-fb0d-49b9-b2b8-f7903309e8ca None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:12.042085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8797a-b9df-49e3-aaff-2b638e3dbb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:12.042728 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8797a-b9df-49e3-aaff-2b638e3dbb3d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:12.043012 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8797a-b9df-49e3-aaff-2b638e3dbb3d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:12.043288 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8797a-b9df-49e3-aaff-2b638e3dbb3d None admin] PATH_INFO: `/v3/projects/274a1dbd7041461a996def94f6dc0656` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:12.044444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8797a-b9df-49e3-aaff-2b638e3dbb3d None admin] RBAC: Authorizing `identity:get_project(project_id=274a1dbd7041461a996def94f6dc0656)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:12.045870 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8797a-b9df-49e3-aaff-2b638e3dbb3d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:12.047532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 566/1116] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:11 2022] GET /identity/v3/projects/274a1dbd7041461a996def94f6dc0656 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:12.053765 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89fc9b-5a5b-49ac-9e18-c0fa5885175d None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:12.146309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:12.150887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e89fc9b-5a5b-49ac-9e18-c0fa5885175d None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:12.183389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d25105-7adb-48f4-ac9f-e76524d11d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:12.184003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d25105-7adb-48f4-ac9f-e76524d11d82 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:12.184284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d25105-7adb-48f4-ac9f-e76524d11d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:12.184556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d25105-7adb-48f4-ac9f-e76524d11d82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:12.185269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d25105-7adb-48f4-ac9f-e76524d11d82 None admin] RBAC: Adding query filter params (role.id=4e002a22056744acb8c40a3236cab404, scope.project.id=274a1dbd7041461a996def94f6dc0656, user.id=19f91bbfbfb24f6d8492ae1553e6ba0d) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:12.185573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d25105-7adb-48f4-ac9f-e76524d11d82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:12.186992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d25105-7adb-48f4-ac9f-e76524d11d82 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:12.192229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 551/1117] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:12 2022] GET /identity/v3/role_assignments?user.id=19f91bbfbfb24f6d8492ae1553e6ba0d&scope.project.id=274a1dbd7041461a996def94f6dc0656&role.id=4e002a22056744acb8c40a3236cab404 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:14.072291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7eb1f-f2bb-4b8b-8d27-90505ef2c38f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:14.072818 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7eb1f-f2bb-4b8b-8d27-90505ef2c38f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:14.072818 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7eb1f-f2bb-4b8b-8d27-90505ef2c38f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:14.074217 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 567/1118] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:14.079699 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69082fc-2b98-40c2-b9fb-0c7f22460adc None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:14.080101 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69082fc-2b98-40c2-b9fb-0c7f22460adc None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:14.080256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69082fc-2b98-40c2-b9fb-0c7f22460adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:14.098321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-d69082fc-2b98-40c2-b9fb-0c7f22460adc None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:14.099505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d69082fc-2b98-40c2-b9fb-0c7f22460adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:14.163692 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 552/1119] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:14.431005 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7eb1f-f2bb-4b8b-8d27-90505ef2c38f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:14.519720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:14.530893 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7eb1f-f2bb-4b8b-8d27-90505ef2c38f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:14.563499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3929009-8e5f-4891-aaf1-2b8ae59497e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:14.564142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3929009-8e5f-4891-aaf1-2b8ae59497e4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:14.564421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3929009-8e5f-4891-aaf1-2b8ae59497e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:14.564695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3929009-8e5f-4891-aaf1-2b8ae59497e4 None admin] PATH_INFO: `/v3/roles/4e002a22056744acb8c40a3236cab404` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:14.566084 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3929009-8e5f-4891-aaf1-2b8ae59497e4 None admin] RBAC: Authorizing `identity:get_role(role_id=4e002a22056744acb8c40a3236cab404)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:14.567451 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3929009-8e5f-4891-aaf1-2b8ae59497e4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:14.568808 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 568/1120] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:14 2022] GET /identity/v3/roles/4e002a22056744acb8c40a3236cab404 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:14.575668 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69082fc-2b98-40c2-b9fb-0c7f22460adc None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:14.662888 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:14.671834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69082fc-2b98-40c2-b9fb-0c7f22460adc None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:14.705408 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8e1ef-9732-4b09-8c81-eced717b7182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:14.705972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8e1ef-9732-4b09-8c81-eced717b7182 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:14.706255 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8e1ef-9732-4b09-8c81-eced717b7182 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:14.706531 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad8e1ef-9732-4b09-8c81-eced717b7182 None admin] PATH_INFO: `/v3/users/19f91bbfbfb24f6d8492ae1553e6ba0d` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:14.707665 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8e1ef-9732-4b09-8c81-eced717b7182 None admin] RBAC: Authorizing `identity:get_user(user_id=19f91bbfbfb24f6d8492ae1553e6ba0d)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:14.709192 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad8e1ef-9732-4b09-8c81-eced717b7182 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:14.710713 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 553/1121] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:14 2022] GET /identity/v3/users/19f91bbfbfb24f6d8492ae1553e6ba0d => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:14.716607 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3929009-8e5f-4891-aaf1-2b8ae59497e4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:14.813269 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3929009-8e5f-4891-aaf1-2b8ae59497e4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:14.847988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be8024-23e1-4781-8d52-ba2fa64a9630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:14.848603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be8024-23e1-4781-8d52-ba2fa64a9630 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:14.848882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be8024-23e1-4781-8d52-ba2fa64a9630 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:14.849153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be8024-23e1-4781-8d52-ba2fa64a9630 None admin] PATH_INFO: `/v3/projects/274a1dbd7041461a996def94f6dc0656` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:14.850246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be8024-23e1-4781-8d52-ba2fa64a9630 None admin] RBAC: Authorizing `identity:get_project(project_id=274a1dbd7041461a996def94f6dc0656)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:14.851710 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be8024-23e1-4781-8d52-ba2fa64a9630 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:14.853551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 569/1122] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:14 2022] GET /identity/v3/projects/274a1dbd7041461a996def94f6dc0656 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:14.859434 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8e1ef-9732-4b09-8c81-eced717b7182 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:14.959972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad8e1ef-9732-4b09-8c81-eced717b7182 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:14.992209 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ed535-bf5b-4761-a206-eefb7c18a5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:14.993390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ed535-bf5b-4761-a206-eefb7c18a5df None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:14.993670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ed535-bf5b-4761-a206-eefb7c18a5df None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:14.993944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ed535-bf5b-4761-a206-eefb7c18a5df None admin] PATH_INFO: `/v3/projects/274a1dbd7041461a996def94f6dc0656/users/19f91bbfbfb24f6d8492ae1553e6ba0d/roles/4e002a22056744acb8c40a3236cab404` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:14.996067 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ed535-bf5b-4761-a206-eefb7c18a5df None admin] RBAC: Authorizing `identity:create_grant(project_id=274a1dbd7041461a996def94f6dc0656, user_id=19f91bbfbfb24f6d8492ae1553e6ba0d, role_id=4e002a22056744acb8c40a3236cab404)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:14.997494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ed535-bf5b-4761-a206-eefb7c18a5df None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:15.010423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 554/1123] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:21:14 2022] PUT /identity/v3/projects/274a1dbd7041461a996def94f6dc0656/users/19f91bbfbfb24f6d8492ae1553e6ba0d/roles/4e002a22056744acb8c40a3236cab404 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:21:16.901389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94dc30-97e5-4055-99ed-25f4df310ed5 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:16.901856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94dc30-97e5-4055-99ed-25f4df310ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:16.901952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94dc30-97e5-4055-99ed-25f4df310ed5 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:16.903297 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 570/1124] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:16.908405 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c1ec7-1ee2-4c9a-bfb2-4e466fddaf2e None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:16.908807 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c1ec7-1ee2-4c9a-bfb2-4e466fddaf2e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:16.908958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137c1ec7-1ee2-4c9a-bfb2-4e466fddaf2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:16.927126 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-137c1ec7-1ee2-4c9a-bfb2-4e466fddaf2e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:16.928345 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-137c1ec7-1ee2-4c9a-bfb2-4e466fddaf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:16.941961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:17.002592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 555/1125] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:17.270561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94dc30-97e5-4055-99ed-25f4df310ed5 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:17.297365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:17.369458 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94dc30-97e5-4055-99ed-25f4df310ed5 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:17.402625 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fd554-f46e-4aa9-94ff-c1bfbafe86d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:17.403239 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fd554-f46e-4aa9-94ff-c1bfbafe86d1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:17.403517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fd554-f46e-4aa9-94ff-c1bfbafe86d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:17.403813 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066fd554-f46e-4aa9-94ff-c1bfbafe86d1 None admin] PATH_INFO: `/v3/roles/4e002a22056744acb8c40a3236cab404` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:17.405163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066fd554-f46e-4aa9-94ff-c1bfbafe86d1 None admin] RBAC: Authorizing `identity:get_role(role_id=4e002a22056744acb8c40a3236cab404)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:17.406510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066fd554-f46e-4aa9-94ff-c1bfbafe86d1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:17.407857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 571/1126] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:17 2022] GET /identity/v3/roles/4e002a22056744acb8c40a3236cab404 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:17.413666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c1ec7-1ee2-4c9a-bfb2-4e466fddaf2e None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:17.514726 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137c1ec7-1ee2-4c9a-bfb2-4e466fddaf2e None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:17.547659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25c7f6-dac1-415b-8096-f10e55dc9878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:17.548221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25c7f6-dac1-415b-8096-f10e55dc9878 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:17.548496 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25c7f6-dac1-415b-8096-f10e55dc9878 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:17.548763 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25c7f6-dac1-415b-8096-f10e55dc9878 None admin] PATH_INFO: `/v3/users/19f91bbfbfb24f6d8492ae1553e6ba0d` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:17.549863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25c7f6-dac1-415b-8096-f10e55dc9878 None admin] RBAC: Authorizing `identity:get_user(user_id=19f91bbfbfb24f6d8492ae1553e6ba0d)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:17.551289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25c7f6-dac1-415b-8096-f10e55dc9878 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:17.552891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 556/1127] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:17 2022] GET /identity/v3/users/19f91bbfbfb24f6d8492ae1553e6ba0d => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:17.558810 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fd554-f46e-4aa9-94ff-c1bfbafe86d1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:17.649911 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:17.656813 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066fd554-f46e-4aa9-94ff-c1bfbafe86d1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:17.690227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4cce1-237a-4297-8a66-2af4c2fd90e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:17.690821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4cce1-237a-4297-8a66-2af4c2fd90e3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:17.691106 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4cce1-237a-4297-8a66-2af4c2fd90e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:17.691383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4cce1-237a-4297-8a66-2af4c2fd90e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:17.756353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4cce1-237a-4297-8a66-2af4c2fd90e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:17.757834 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4cce1-237a-4297-8a66-2af4c2fd90e3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:17.845946 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:17.853679 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 572/1128] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:17.859740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25c7f6-dac1-415b-8096-f10e55dc9878 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:17.878897 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:17.963910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25c7f6-dac1-415b-8096-f10e55dc9878 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:17.999279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd7884-ba35-4558-a49b-e3d1c40efb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:17.999926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd7884-ba35-4558-a49b-e3d1c40efb94 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:18.000202 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd7884-ba35-4558-a49b-e3d1c40efb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:18.000469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd7884-ba35-4558-a49b-e3d1c40efb94 None admin] PATH_INFO: `/v3/projects/274a1dbd7041461a996def94f6dc0656` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:18.001545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd7884-ba35-4558-a49b-e3d1c40efb94 None admin] RBAC: Authorizing `identity:get_project(project_id=274a1dbd7041461a996def94f6dc0656)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:18.002967 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd7884-ba35-4558-a49b-e3d1c40efb94 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:18.004611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 557/1129] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:17 2022] GET /identity/v3/projects/274a1dbd7041461a996def94f6dc0656 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:18.011071 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4cce1-237a-4297-8a66-2af4c2fd90e3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:18.112348 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4cce1-237a-4297-8a66-2af4c2fd90e3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:18.145100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc7878-f166-4c30-b5c6-9346b29c04ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:18.145695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc7878-f166-4c30-b5c6-9346b29c04ec None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:18.145977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc7878-f166-4c30-b5c6-9346b29c04ec None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:18.146253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc7878-f166-4c30-b5c6-9346b29c04ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:18.146971 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc7878-f166-4c30-b5c6-9346b29c04ec None admin] RBAC: Adding query filter params (role.id=4e002a22056744acb8c40a3236cab404, scope.project.id=274a1dbd7041461a996def94f6dc0656, user.id=19f91bbfbfb24f6d8492ae1553e6ba0d) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:18.147279 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc7878-f166-4c30-b5c6-9346b29c04ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:18.149281 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc7878-f166-4c30-b5c6-9346b29c04ec None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:18.154938 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 573/1130] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:18 2022] GET /identity/v3/role_assignments?user.id=19f91bbfbfb24f6d8492ae1553e6ba0d&scope.project.id=274a1dbd7041461a996def94f6dc0656&role.id=4e002a22056744acb8c40a3236cab404 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:20.056248 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14866c33-cac4-4c2d-acf5-192897d4130e None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:20.056801 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14866c33-cac4-4c2d-acf5-192897d4130e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:20.056801 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14866c33-cac4-4c2d-acf5-192897d4130e None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:20.058136 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 558/1131] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:20.063194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dc78f-760f-4f6f-af9f-32b807d0e6b7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:20.063654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dc78f-760f-4f6f-af9f-32b807d0e6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:20.063768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dc78f-760f-4f6f-af9f-32b807d0e6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:20.082061 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6d9dc78f-760f-4f6f-af9f-32b807d0e6b7 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:20.083236 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6d9dc78f-760f-4f6f-af9f-32b807d0e6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:20.134685 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:20.148214 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 574/1132] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:20.417123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14866c33-cac4-4c2d-acf5-192897d4130e None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:20.500946 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:20.516626 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14866c33-cac4-4c2d-acf5-192897d4130e None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:20.549927 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008cfe1-995e-4cef-9612-753593466b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:20.550556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008cfe1-995e-4cef-9612-753593466b7f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:20.550836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008cfe1-995e-4cef-9612-753593466b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:20.551112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008cfe1-995e-4cef-9612-753593466b7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:20.552223 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008cfe1-995e-4cef-9612-753593466b7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:20.553662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008cfe1-995e-4cef-9612-753593466b7f None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:20.555271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 559/1133] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:20.561127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9dc78f-760f-4f6f-af9f-32b807d0e6b7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:20.657049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9dc78f-760f-4f6f-af9f-32b807d0e6b7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:20.690093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218663d0-28df-4439-9095-785eddbcf0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:20.690626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218663d0-28df-4439-9095-785eddbcf0d0 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:20.690908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218663d0-28df-4439-9095-785eddbcf0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:20.691180 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218663d0-28df-4439-9095-785eddbcf0d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:20.691850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218663d0-28df-4439-9095-785eddbcf0d0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:20.693275 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218663d0-28df-4439-9095-785eddbcf0d0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:20.726629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 575/1134] 10.222.0.58 () {60 vars in 1141 bytes} [Wed Aug 24 12:21:20 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:21:22.647476 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4ec58-f970-4c1f-a400-bbbd588ede1b None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:22.648083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4ec58-f970-4c1f-a400-bbbd588ede1b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:22.648083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4ec58-f970-4c1f-a400-bbbd588ede1b None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:22.649428 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 560/1135] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:22.654617 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897fdb5-b494-4b7f-9a7e-6cd504cce5e0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:22.655080 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897fdb5-b494-4b7f-9a7e-6cd504cce5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:22.655185 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897fdb5-b494-4b7f-9a7e-6cd504cce5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:22.673423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5897fdb5-b494-4b7f-9a7e-6cd504cce5e0 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:22.674607 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5897fdb5-b494-4b7f-9a7e-6cd504cce5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:22.710246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:22.744319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 576/1136] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:23.017790 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4ec58-f970-4c1f-a400-bbbd588ede1b None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:23.117909 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4ec58-f970-4c1f-a400-bbbd588ede1b None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:23.151197 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cafefc-4789-466b-a153-d5c519e4c022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:23.151833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cafefc-4789-466b-a153-d5c519e4c022 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:23.152111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cafefc-4789-466b-a153-d5c519e4c022 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:23.152379 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cafefc-4789-466b-a153-d5c519e4c022 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:23.153466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cafefc-4789-466b-a153-d5c519e4c022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:23.154878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cafefc-4789-466b-a153-d5c519e4c022 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:23.156443 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 561/1137] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:21:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:23.162141 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897fdb5-b494-4b7f-9a7e-6cd504cce5e0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:23.260388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897fdb5-b494-4b7f-9a7e-6cd504cce5e0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:23.295658 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca621d-221d-4e7d-b73b-fde5ca817c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:23.296206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca621d-221d-4e7d-b73b-fde5ca817c82 None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:23.296488 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca621d-221d-4e7d-b73b-fde5ca817c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:23.296762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca621d-221d-4e7d-b73b-fde5ca817c82 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:23.297409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca621d-221d-4e7d-b73b-fde5ca817c82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:23.298839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca621d-221d-4e7d-b73b-fde5ca817c82 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:23.319806 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 577/1138] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:21:23 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:21:25.284894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc882c-6a33-47c1-b5a5-4d0d3e6f16f7 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:25.285414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc882c-6a33-47c1-b5a5-4d0d3e6f16f7 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:25.285414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc882c-6a33-47c1-b5a5-4d0d3e6f16f7 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:25.286703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 562/1139] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:25.291993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c60bb-d979-455b-abc3-420c76245302 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:25.292389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c60bb-d979-455b-abc3-420c76245302 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:25.292554 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c60bb-d979-455b-abc3-420c76245302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:25.310729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-540c60bb-d979-455b-abc3-420c76245302 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:25.311954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-540c60bb-d979-455b-abc3-420c76245302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:25.332523 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:25.378285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 578/1140] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:25.644042 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc882c-6a33-47c1-b5a5-4d0d3e6f16f7 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:25.668939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:25.744846 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc882c-6a33-47c1-b5a5-4d0d3e6f16f7 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:25.776975 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c89d68-e6ad-42c1-81c0-1468002f9e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:25.777603 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c89d68-e6ad-42c1-81c0-1468002f9e1e None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:25.777891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c89d68-e6ad-42c1-81c0-1468002f9e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:25.778170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c89d68-e6ad-42c1-81c0-1468002f9e1e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:25.788488 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c89d68-e6ad-42c1-81c0-1468002f9e1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:25.793894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-b5c89d68-e6ad-42c1-81c0-1468002f9e1e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:25.793894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:25.793894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:25.793894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:25.793894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:25.793894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:25.793894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:25.793894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:25.793894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:25.793894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:25.793894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:25.793894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:25.793894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:25.793894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:25.793894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:25.793894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:25.795104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:25.795104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:25.795104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:25.795104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:25.795104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:25.795104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:25.795104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:25.795104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:25.795104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:25.795104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:25.795104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:25.795104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:25.795104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:25.795104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:25.795104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:25.795104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:25.796315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:25.796315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:25.796315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:25.796315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:25.796315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:25.796315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:25.796315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:25.796315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:25.796315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:25.796315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:25.796315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:25.796315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:21:25.796315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 563/1141] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:21:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:25.801040 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c60bb-d979-455b-abc3-420c76245302 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:25.897699 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c60bb-d979-455b-abc3-420c76245302 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:25.931144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a96c3-bc6c-4b9f-89c4-e80798e115f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:25.931770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a96c3-bc6c-4b9f-89c4-e80798e115f3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:25.932052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a96c3-bc6c-4b9f-89c4-e80798e115f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:25.932331 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a96c3-bc6c-4b9f-89c4-e80798e115f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:25.932990 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a96c3-bc6c-4b9f-89c4-e80798e115f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:25.933296 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a96c3-bc6c-4b9f-89c4-e80798e115f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:25.934665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a96c3-bc6c-4b9f-89c4-e80798e115f3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:25.942433 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 579/1142] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:21:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:25.948519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c89d68-e6ad-42c1-81c0-1468002f9e1e None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:26.027921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:26.046089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c89d68-e6ad-42c1-81c0-1468002f9e1e None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:26.079329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f8817-8b85-468f-8fa5-7f472878f9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:26.079970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f8817-8b85-468f-8fa5-7f472878f9cf None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:26.080248 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f8817-8b85-468f-8fa5-7f472878f9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:26.080516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f8817-8b85-468f-8fa5-7f472878f9cf None admin] PATH_INFO: `/v3/users/f47f55b57ded443da16a0d3690cabf65` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:26.093550 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f8817-8b85-468f-8fa5-7f472878f9cf None admin] RBAC: Authorizing `identity:get_user(user_id=f47f55b57ded443da16a0d3690cabf65)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:26.094981 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f8817-8b85-468f-8fa5-7f472878f9cf None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:26.096584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 564/1143] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:25 2022] GET /identity/v3/users/f47f55b57ded443da16a0d3690cabf65 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:26.102996 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a96c3-bc6c-4b9f-89c4-e80798e115f3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:26.122049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:26.202192 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a96c3-bc6c-4b9f-89c4-e80798e115f3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:26.235073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244843d-b255-456e-952c-254cf573c43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:26.235691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244843d-b255-456e-952c-254cf573c43f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:26.235974 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244843d-b255-456e-952c-254cf573c43f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:26.236247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244843d-b255-456e-952c-254cf573c43f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:26.301206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244843d-b255-456e-952c-254cf573c43f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:26.302642 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244843d-b255-456e-952c-254cf573c43f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:26.319704 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:26.401261 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 580/1144] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:26.407240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f8817-8b85-468f-8fa5-7f472878f9cf None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:26.504166 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f8817-8b85-468f-8fa5-7f472878f9cf None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:26.537675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f69c21-8787-4af3-9578-038a62b6202f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:26.538296 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f69c21-8787-4af3-9578-038a62b6202f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:26.538572 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f69c21-8787-4af3-9578-038a62b6202f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:26.538867 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f69c21-8787-4af3-9578-038a62b6202f None admin] PATH_INFO: `/v3/projects/b078ac68892848fbad11a423af4e97a8` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:26.540071 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f69c21-8787-4af3-9578-038a62b6202f None admin] RBAC: Authorizing `identity:get_project(project_id=b078ac68892848fbad11a423af4e97a8)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:26.541490 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f69c21-8787-4af3-9578-038a62b6202f None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:26.543110 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 565/1145] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:26 2022] GET /identity/v3/projects/b078ac68892848fbad11a423af4e97a8 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:26.549375 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244843d-b255-456e-952c-254cf573c43f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:26.645443 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244843d-b255-456e-952c-254cf573c43f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:26.662585 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:26.680197 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a581c-764a-43f9-8a78-dd5c256a45d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:26.680787 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a581c-764a-43f9-8a78-dd5c256a45d0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:26.681068 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a581c-764a-43f9-8a78-dd5c256a45d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:26.681343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a581c-764a-43f9-8a78-dd5c256a45d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:26.682064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a581c-764a-43f9-8a78-dd5c256a45d0 None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=b078ac68892848fbad11a423af4e97a8, user.id=f47f55b57ded443da16a0d3690cabf65) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:26.682370 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a581c-764a-43f9-8a78-dd5c256a45d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:26.683822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a581c-764a-43f9-8a78-dd5c256a45d0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:26.689030 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 581/1146] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:26 2022] GET /identity/v3/role_assignments?user.id=f47f55b57ded443da16a0d3690cabf65&scope.project.id=b078ac68892848fbad11a423af4e97a8&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:28.556129 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f487c4-68b0-4dd4-8448-926e87818e37 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:28.556742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f487c4-68b0-4dd4-8448-926e87818e37 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:28.556742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f487c4-68b0-4dd4-8448-926e87818e37 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:28.558015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 566/1147] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:28.563362 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54ae17-b176-422b-9b3a-971c52b6d6ec None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:28.563829 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54ae17-b176-422b-9b3a-971c52b6d6ec None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:28.563930 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54ae17-b176-422b-9b3a-971c52b6d6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:28.582349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7e54ae17-b176-422b-9b3a-971c52b6d6ec None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:28.583536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7e54ae17-b176-422b-9b3a-971c52b6d6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:28.649334 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 582/1148] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:28.920453 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f487c4-68b0-4dd4-8448-926e87818e37 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:28.932009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:29.021365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f487c4-68b0-4dd4-8448-926e87818e37 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:29.053825 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abff026-baa9-4d16-aaca-ad11965e191f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:29.054947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abff026-baa9-4d16-aaca-ad11965e191f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:29.055228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abff026-baa9-4d16-aaca-ad11965e191f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:29.055502 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abff026-baa9-4d16-aaca-ad11965e191f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:29.065299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abff026-baa9-4d16-aaca-ad11965e191f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:29.070573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-1abff026-baa9-4d16-aaca-ad11965e191f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:29.070573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:29.070573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:29.070573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:29.070573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:29.070573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:29.070573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:29.070573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:29.070573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:29.070573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:29.070573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:29.070573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:29.070573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:29.070573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:29.070573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:29.070573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:29.071941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:29.071941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:29.071941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:29.071941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:29.071941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:29.071941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:29.071941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:29.071941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:29.071941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:29.071941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:29.071941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:29.071941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:29.071941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:29.071941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:29.071941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:29.071941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:29.073141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:29.073141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:29.073141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:29.073141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:29.073141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:29.073141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:29.073141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:29.073141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:29.073141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:29.073141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:29.073141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:29.073141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:21:29.073141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 567/1149] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:21:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:29.077952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54ae17-b176-422b-9b3a-971c52b6d6ec None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:29.171739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54ae17-b176-422b-9b3a-971c52b6d6ec None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:29.184634 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:29.205904 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bb619-5a82-44b4-80e4-fec57180b7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:29.206492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bb619-5a82-44b4-80e4-fec57180b7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:29.206768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bb619-5a82-44b4-80e4-fec57180b7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:29.207038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bb619-5a82-44b4-80e4-fec57180b7c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:29.207717 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bb619-5a82-44b4-80e4-fec57180b7c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:29.208018 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bb619-5a82-44b4-80e4-fec57180b7c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:29.209375 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4bb619-5a82-44b4-80e4-fec57180b7c3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:29.217185 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 583/1150] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:21:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:29.222912 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abff026-baa9-4d16-aaca-ad11965e191f None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:29.287717 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:29.320434 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abff026-baa9-4d16-aaca-ad11965e191f None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:29.353502 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3a8dd-bf64-4db5-9200-773c171e3e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:29.354114 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3a8dd-bf64-4db5-9200-773c171e3e7d None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:29.354394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3a8dd-bf64-4db5-9200-773c171e3e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:29.354669 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf3a8dd-bf64-4db5-9200-773c171e3e7d None admin] PATH_INFO: `/v3/users/f47f55b57ded443da16a0d3690cabf65` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:29.355806 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3a8dd-bf64-4db5-9200-773c171e3e7d None admin] RBAC: Authorizing `identity:get_user(user_id=f47f55b57ded443da16a0d3690cabf65)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:29.357242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf3a8dd-bf64-4db5-9200-773c171e3e7d None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:29.358801 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 568/1151] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:29 2022] GET /identity/v3/users/f47f55b57ded443da16a0d3690cabf65 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:29.364640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bb619-5a82-44b4-80e4-fec57180b7c3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:29.462489 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4bb619-5a82-44b4-80e4-fec57180b7c3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:29.495230 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e38e1-dedf-49ec-8ecf-af1867fe6e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:29.495862 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e38e1-dedf-49ec-8ecf-af1867fe6e89 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:29.496140 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e38e1-dedf-49ec-8ecf-af1867fe6e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:29.496410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e38e1-dedf-49ec-8ecf-af1867fe6e89 None admin] PATH_INFO: `/v3/projects/b078ac68892848fbad11a423af4e97a8` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:29.497492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e38e1-dedf-49ec-8ecf-af1867fe6e89 None admin] RBAC: Authorizing `identity:get_project(project_id=b078ac68892848fbad11a423af4e97a8)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:29.498904 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e38e1-dedf-49ec-8ecf-af1867fe6e89 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:29.500536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 584/1152] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:29 2022] GET /identity/v3/projects/b078ac68892848fbad11a423af4e97a8 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:29.506354 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3a8dd-bf64-4db5-9200-773c171e3e7d None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:29.603778 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf3a8dd-bf64-4db5-9200-773c171e3e7d None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:29.636712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1b3c8-3f84-4b14-8f30-8a477f74ec77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:29.637358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1b3c8-3f84-4b14-8f30-8a477f74ec77 None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:29.637638 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1b3c8-3f84-4b14-8f30-8a477f74ec77 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:29.637908 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1b3c8-3f84-4b14-8f30-8a477f74ec77 None admin] PATH_INFO: `/v3/projects/b078ac68892848fbad11a423af4e97a8/users/f47f55b57ded443da16a0d3690cabf65/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:29.639797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1b3c8-3f84-4b14-8f30-8a477f74ec77 None admin] RBAC: Authorizing `identity:create_grant(project_id=b078ac68892848fbad11a423af4e97a8, user_id=f47f55b57ded443da16a0d3690cabf65, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:29.641235 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1b3c8-3f84-4b14-8f30-8a477f74ec77 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:29.644176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:29.656333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 569/1153] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:21:29 2022] PUT /identity/v3/projects/b078ac68892848fbad11a423af4e97a8/users/f47f55b57ded443da16a0d3690cabf65/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:21:31.681998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b66a1a-f351-479f-bfa7-06f5e212e1d2 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:31.682602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b66a1a-f351-479f-bfa7-06f5e212e1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:31.682602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b66a1a-f351-479f-bfa7-06f5e212e1d2 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:31.684082 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 585/1154] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:31.689195 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327ce8f-9042-4a8f-ae1b-b3b6c76ff873 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:31.689600 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327ce8f-9042-4a8f-ae1b-b3b6c76ff873 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:31.689738 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327ce8f-9042-4a8f-ae1b-b3b6c76ff873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:31.708239 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-c327ce8f-9042-4a8f-ae1b-b3b6c76ff873 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:31.709419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c327ce8f-9042-4a8f-ae1b-b3b6c76ff873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:31.774995 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 570/1155] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:32.043615 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b66a1a-f351-479f-bfa7-06f5e212e1d2 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:32.088641 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:32.146323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b66a1a-f351-479f-bfa7-06f5e212e1d2 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:32.178616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3406f3-8b70-4422-8ba4-12a8836d5100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:32.179923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3406f3-8b70-4422-8ba4-12a8836d5100 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:32.180211 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3406f3-8b70-4422-8ba4-12a8836d5100 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:32.180483 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3406f3-8b70-4422-8ba4-12a8836d5100 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:32.191943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3406f3-8b70-4422-8ba4-12a8836d5100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:32.197603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-1d3406f3-8b70-4422-8ba4-12a8836d5100 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:32.197603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:32.197603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:32.197603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:32.197603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:32.197603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:32.197603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:32.197603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:32.197603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:32.197603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:32.197603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:32.197603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:32.197603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:32.197603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:32.197603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:32.197603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:32.198757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:32.198757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:32.198757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:32.198757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:32.198757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:32.198757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:32.198757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:32.198757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:32.198757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:32.198757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:32.198757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:32.198757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:32.198757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:32.198757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:32.198757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:32.198757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:32.200033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:32.200033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:32.200033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:32.200033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:32.200033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:32.200033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:32.200033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:32.200033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:32.200033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:32.200033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:32.200033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:32.200033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:21:32.200033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 586/1156] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:21:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:32.205014 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327ce8f-9042-4a8f-ae1b-b3b6c76ff873 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:32.300237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327ce8f-9042-4a8f-ae1b-b3b6c76ff873 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:32.325217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:32.334271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81442582-7599-4d3d-9e22-2ea5ddd58988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:32.334873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81442582-7599-4d3d-9e22-2ea5ddd58988 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:32.335155 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81442582-7599-4d3d-9e22-2ea5ddd58988 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:32.335496 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81442582-7599-4d3d-9e22-2ea5ddd58988 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:32.336244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81442582-7599-4d3d-9e22-2ea5ddd58988 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:32.336551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81442582-7599-4d3d-9e22-2ea5ddd58988 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:32.337912 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81442582-7599-4d3d-9e22-2ea5ddd58988 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:32.345866 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 571/1157] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:21:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:32.352053 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3406f3-8b70-4422-8ba4-12a8836d5100 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:32.465581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3406f3-8b70-4422-8ba4-12a8836d5100 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:32.467075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:32.498338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7ec8f-3326-41a3-b3e0-dbb9055c779d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:32.498945 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7ec8f-3326-41a3-b3e0-dbb9055c779d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:32.499220 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7ec8f-3326-41a3-b3e0-dbb9055c779d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:32.499492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7ec8f-3326-41a3-b3e0-dbb9055c779d None admin] PATH_INFO: `/v3/users/f47f55b57ded443da16a0d3690cabf65` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:32.500605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7ec8f-3326-41a3-b3e0-dbb9055c779d None admin] RBAC: Authorizing `identity:get_user(user_id=f47f55b57ded443da16a0d3690cabf65)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:32.502028 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7ec8f-3326-41a3-b3e0-dbb9055c779d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:32.503591 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 587/1158] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:32 2022] GET /identity/v3/users/f47f55b57ded443da16a0d3690cabf65 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:32.510334 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81442582-7599-4d3d-9e22-2ea5ddd58988 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:32.607169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81442582-7599-4d3d-9e22-2ea5ddd58988 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:32.639180 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fded27-0d76-41bb-96ee-b9dd742af4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:32.639802 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fded27-0d76-41bb-96ee-b9dd742af4ab None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:32.640084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fded27-0d76-41bb-96ee-b9dd742af4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:32.640353 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fded27-0d76-41bb-96ee-b9dd742af4ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:32.693050 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:32.708344 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fded27-0d76-41bb-96ee-b9dd742af4ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:32.709798 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fded27-0d76-41bb-96ee-b9dd742af4ab None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:32.804698 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 572/1159] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:32.811073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7ec8f-3326-41a3-b3e0-dbb9055c779d None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:32.915861 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7ec8f-3326-41a3-b3e0-dbb9055c779d None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:32.948854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc85bd-8a76-4b07-8fe4-ee1649c2f9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:32.949465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc85bd-8a76-4b07-8fe4-ee1649c2f9f8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:32.949994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc85bd-8a76-4b07-8fe4-ee1649c2f9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:32.950262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc85bd-8a76-4b07-8fe4-ee1649c2f9f8 None admin] PATH_INFO: `/v3/projects/b078ac68892848fbad11a423af4e97a8` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:32.951359 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc85bd-8a76-4b07-8fe4-ee1649c2f9f8 None admin] RBAC: Authorizing `identity:get_project(project_id=b078ac68892848fbad11a423af4e97a8)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:32.952803 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc85bd-8a76-4b07-8fe4-ee1649c2f9f8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:32.954428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 588/1160] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:32 2022] GET /identity/v3/projects/b078ac68892848fbad11a423af4e97a8 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:32.960891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fded27-0d76-41bb-96ee-b9dd742af4ab None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:33.045693 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:33.058421 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fded27-0d76-41bb-96ee-b9dd742af4ab None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:33.091421 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0276bb-82de-447b-82e5-08b0e5b73274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:33.092047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0276bb-82de-447b-82e5-08b0e5b73274 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:33.092330 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0276bb-82de-447b-82e5-08b0e5b73274 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:33.092607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0276bb-82de-447b-82e5-08b0e5b73274 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:33.093327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0276bb-82de-447b-82e5-08b0e5b73274 None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=b078ac68892848fbad11a423af4e97a8, user.id=f47f55b57ded443da16a0d3690cabf65) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:33.093637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0276bb-82de-447b-82e5-08b0e5b73274 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:33.095061 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0276bb-82de-447b-82e5-08b0e5b73274 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:33.100728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 573/1161] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:32 2022] GET /identity/v3/role_assignments?user.id=f47f55b57ded443da16a0d3690cabf65&scope.project.id=b078ac68892848fbad11a423af4e97a8&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:35.015820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8eaff-b9c4-4908-bdaa-8800fffb3e67 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:35.016359 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8eaff-b9c4-4908-bdaa-8800fffb3e67 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:35.016359 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8eaff-b9c4-4908-bdaa-8800fffb3e67 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:35.017704 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 589/1162] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:35.023491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24132c93-f251-4530-b2a2-34c538202550 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:35.023930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24132c93-f251-4530-b2a2-34c538202550 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:35.024073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24132c93-f251-4530-b2a2-34c538202550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:35.042464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-24132c93-f251-4530-b2a2-34c538202550 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:35.043683 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-24132c93-f251-4530-b2a2-34c538202550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:35.109561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 574/1163] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:35.376394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8eaff-b9c4-4908-bdaa-8800fffb3e67 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:35.416969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:35.478140 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8eaff-b9c4-4908-bdaa-8800fffb3e67 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:35.510945 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73515b-7549-4991-8613-55dfdeaf5f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:35.511542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73515b-7549-4991-8613-55dfdeaf5f8c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:35.511850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73515b-7549-4991-8613-55dfdeaf5f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:35.512125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73515b-7549-4991-8613-55dfdeaf5f8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:35.577152 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73515b-7549-4991-8613-55dfdeaf5f8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:35.578595 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73515b-7549-4991-8613-55dfdeaf5f8c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:35.615101 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:35.674848 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 590/1164] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:35.681535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24132c93-f251-4530-b2a2-34c538202550 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:35.760929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:35.778827 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24132c93-f251-4530-b2a2-34c538202550 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:35.819560 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a871c-6d97-4c63-9a82-aa2d3f6836e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:35.821013 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a871c-6d97-4c63-9a82-aa2d3f6836e6 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:35.821298 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a871c-6d97-4c63-9a82-aa2d3f6836e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:35.821569 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a871c-6d97-4c63-9a82-aa2d3f6836e6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:35.828349 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a871c-6d97-4c63-9a82-aa2d3f6836e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:35.829789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a871c-6d97-4c63-9a82-aa2d3f6836e6 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-708a871c-6d97-4c63-9a82-aa2d3f6836e6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:35.840829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:35.842119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:35.842119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:35.842119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:35.842119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:35.842119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:35.842119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:35.842119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:21:35.842119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:21:35.842119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:21:35.842119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:21:35.842119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:35.842119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:35.842119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:35.842119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:35.842119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:35.842119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:35.843286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:35.843286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:35.843286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:35.843286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:35.843286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:35.843286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:35.843286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:35.843286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:35.843286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:35.843286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:35.843286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 12:21:35.843286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:21:35.843286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 12:21:35.843286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:21:35.843286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 575/1165] 10.222.0.58 () {56 vars in 1117 bytes} [Wed Aug 24 12:21:35 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:35.847982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73515b-7549-4991-8613-55dfdeaf5f8c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:35.946307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73515b-7549-4991-8613-55dfdeaf5f8c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:35.979179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3442af97-0c42-446e-9285-b68984bbce3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:35.979789 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442af97-0c42-446e-9285-b68984bbce3c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:35.980064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442af97-0c42-446e-9285-b68984bbce3c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:35.980328 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3442af97-0c42-446e-9285-b68984bbce3c None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:35.980946 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442af97-0c42-446e-9285-b68984bbce3c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:35.981247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442af97-0c42-446e-9285-b68984bbce3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:35.982587 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3442af97-0c42-446e-9285-b68984bbce3c None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:35.989363 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 591/1166] 10.222.0.58 () {56 vars in 1115 bytes} [Wed Aug 24 12:21:35 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:37.862141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8168f-4ccc-487b-911b-e7c6d123faf8 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:37.862709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8168f-4ccc-487b-911b-e7c6d123faf8 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:37.862709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8168f-4ccc-487b-911b-e7c6d123faf8 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:37.864067 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 576/1167] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:37.870143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222a9e6-3c45-4034-bbce-9f98465c3936 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:37.870561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222a9e6-3c45-4034-bbce-9f98465c3936 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:37.870673 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222a9e6-3c45-4034-bbce-9f98465c3936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:37.879418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:37.890051 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c222a9e6-3c45-4034-bbce-9f98465c3936 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:37.891234 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c222a9e6-3c45-4034-bbce-9f98465c3936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:37.957048 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 592/1168] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:38.225136 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8168f-4ccc-487b-911b-e7c6d123faf8 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:38.324752 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8168f-4ccc-487b-911b-e7c6d123faf8 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:38.357347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03be7a5-823a-45ed-b132-889519ed64ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:38.357943 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03be7a5-823a-45ed-b132-889519ed64ae None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:38.358224 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03be7a5-823a-45ed-b132-889519ed64ae None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:38.358498 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03be7a5-823a-45ed-b132-889519ed64ae None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:38.359159 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03be7a5-823a-45ed-b132-889519ed64ae None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:38.360553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03be7a5-823a-45ed-b132-889519ed64ae None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:38.376995 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 577/1169] 10.222.0.58 () {60 vars in 1138 bytes} [Wed Aug 24 12:21:38 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:21:40.288122 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18126a-abca-42f6-a72c-96560ae899c6 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:40.289192 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18126a-abca-42f6-a72c-96560ae899c6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:40.289192 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18126a-abca-42f6-a72c-96560ae899c6 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:40.290487 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 593/1170] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:40.295852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b235146-7da8-4b44-920d-62298d88867d None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:40.296284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b235146-7da8-4b44-920d-62298d88867d None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:40.296406 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b235146-7da8-4b44-920d-62298d88867d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:40.302560 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:40.315805 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-9b235146-7da8-4b44-920d-62298d88867d None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:40.316987 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-9b235146-7da8-4b44-920d-62298d88867d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:40.381417 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 578/1171] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:40.650562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18126a-abca-42f6-a72c-96560ae899c6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:40.749889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18126a-abca-42f6-a72c-96560ae899c6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:40.779008 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:40.784464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c08978-2c57-42c0-93ac-1ff4fbe36fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:40.785093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c08978-2c57-42c0-93ac-1ff4fbe36fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:40.785373 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c08978-2c57-42c0-93ac-1ff4fbe36fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:40.785647 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c08978-2c57-42c0-93ac-1ff4fbe36fe0 None admin] PATH_INFO: `/v3/domains/714a0561face4c9c823a2b0027e85873` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:40.795886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c08978-2c57-42c0-93ac-1ff4fbe36fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=714a0561face4c9c823a2b0027e85873)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:40.797328 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c08978-2c57-42c0-93ac-1ff4fbe36fe0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:40.798913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 594/1172] 10.222.0.58 () {56 vars in 1183 bytes} [Wed Aug 24 12:21:40 2022] GET /identity/v3/domains/714a0561face4c9c823a2b0027e85873 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:40.804772 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b235146-7da8-4b44-920d-62298d88867d None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:40.902168 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b235146-7da8-4b44-920d-62298d88867d None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:40.930209 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:40.935283 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29523f89-d66d-499e-82fe-d54607d04507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:40.935924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29523f89-d66d-499e-82fe-d54607d04507 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:40.936202 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29523f89-d66d-499e-82fe-d54607d04507 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:40.936471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29523f89-d66d-499e-82fe-d54607d04507 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:40.937108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29523f89-d66d-499e-82fe-d54607d04507 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:40.938528 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29523f89-d66d-499e-82fe-d54607d04507 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:40.964360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 579/1173] 10.222.0.58 () {60 vars in 1141 bytes} [Wed Aug 24 12:21:40 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:21:42.877474 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af7626-6ece-43f2-a1c9-2190a5c65e22 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:42.877994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af7626-6ece-43f2-a1c9-2190a5c65e22 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:42.877994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af7626-6ece-43f2-a1c9-2190a5c65e22 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:42.879363 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 595/1174] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:42.884821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5d423-4e39-4157-95be-1205e3fbb1da None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:42.885213 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5d423-4e39-4157-95be-1205e3fbb1da None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:42.885378 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5d423-4e39-4157-95be-1205e3fbb1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:42.903145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-0dc5d423-4e39-4157-95be-1205e3fbb1da None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:42.904370 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-0dc5d423-4e39-4157-95be-1205e3fbb1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:42.969433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 580/1175] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:43.236268 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af7626-6ece-43f2-a1c9-2190a5c65e22 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:43.334808 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af7626-6ece-43f2-a1c9-2190a5c65e22 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:43.367731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5460591-4101-4128-90f7-773ed1312f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:43.368336 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5460591-4101-4128-90f7-773ed1312f4d None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:43.368611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5460591-4101-4128-90f7-773ed1312f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:43.368878 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5460591-4101-4128-90f7-773ed1312f4d None admin] PATH_INFO: `/v3/domains/714a0561face4c9c823a2b0027e85873` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:43.369961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5460591-4101-4128-90f7-773ed1312f4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=714a0561face4c9c823a2b0027e85873)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:43.371379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5460591-4101-4128-90f7-773ed1312f4d None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:43.372946 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 596/1176] 10.222.0.58 () {56 vars in 1183 bytes} [Wed Aug 24 12:21:43 2022] GET /identity/v3/domains/714a0561face4c9c823a2b0027e85873 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:43.378594 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5d423-4e39-4157-95be-1205e3fbb1da None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:43.475227 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5d423-4e39-4157-95be-1205e3fbb1da None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:43.489400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:43.509188 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb15987-627e-473c-ab56-ea9d4bef604a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:43.509786 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb15987-627e-473c-ab56-ea9d4bef604a None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:43.510067 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb15987-627e-473c-ab56-ea9d4bef604a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:43.510341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb15987-627e-473c-ab56-ea9d4bef604a None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:43.510977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb15987-627e-473c-ab56-ea9d4bef604a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:43.512437 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb15987-627e-473c-ab56-ea9d4bef604a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:43.533705 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 581/1177] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:21:43 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:21:45.471219 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759e456-6f6b-4865-9df9-5f12418a1c59 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:45.471798 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759e456-6f6b-4865-9df9-5f12418a1c59 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:45.471798 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759e456-6f6b-4865-9df9-5f12418a1c59 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:45.473161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 597/1178] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:45.478330 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51d271-5b38-47e5-875f-ed4e1926e287 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:45.478760 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51d271-5b38-47e5-875f-ed4e1926e287 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:45.478882 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51d271-5b38-47e5-875f-ed4e1926e287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:45.497171 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-3e51d271-5b38-47e5-875f-ed4e1926e287 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:45.498352 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-3e51d271-5b38-47e5-875f-ed4e1926e287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:45.562800 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 582/1179] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:45.832421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759e456-6f6b-4865-9df9-5f12418a1c59 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:45.892514 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:45.935110 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759e456-6f6b-4865-9df9-5f12418a1c59 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:45.967054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1950edd-42b2-4917-a1c3-cd73b31fc974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:45.967697 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1950edd-42b2-4917-a1c3-cd73b31fc974 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:45.968010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1950edd-42b2-4917-a1c3-cd73b31fc974 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:45.968284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1950edd-42b2-4917-a1c3-cd73b31fc974 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:45.978999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1950edd-42b2-4917-a1c3-cd73b31fc974 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:45.983691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-d1950edd-42b2-4917-a1c3-cd73b31fc974 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:45.983691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:45.983691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:45.983691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:45.983691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:45.983691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:45.983691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:45.983691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:45.983691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:45.983691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:45.983691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:45.983691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:45.983691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:45.983691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:45.983691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:45.983691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:45.984837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:45.984837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:45.984837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:45.984837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:45.984837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:45.984837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:45.984837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:45.984837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:45.984837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:45.984837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:45.984837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:45.984837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:45.984837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:45.984837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:45.984837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:45.984837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:45.986031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:45.986031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:45.986031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:45.986031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:45.986031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:45.986031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:45.986031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:45.986031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:45.986031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:45.986031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:45.986031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:45.986031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:21:45.986031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 598/1180] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:21:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:45.990778 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51d271-5b38-47e5-875f-ed4e1926e287 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:46.084632 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:46.087570 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51d271-5b38-47e5-875f-ed4e1926e287 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:46.122267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7f1f0-8e04-45c2-898e-c58ac1946a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:46.122865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7f1f0-8e04-45c2-898e-c58ac1946a97 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:46.123143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7f1f0-8e04-45c2-898e-c58ac1946a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:46.123411 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7f1f0-8e04-45c2-898e-c58ac1946a97 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:46.124087 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7f1f0-8e04-45c2-898e-c58ac1946a97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:46.124389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7f1f0-8e04-45c2-898e-c58ac1946a97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:46.125751 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7f1f0-8e04-45c2-898e-c58ac1946a97 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:46.133505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 583/1181] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:21:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:46.139208 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1950edd-42b2-4917-a1c3-cd73b31fc974 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:46.238181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1950edd-42b2-4917-a1c3-cd73b31fc974 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:46.263692 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:46.278227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaf9bf-a22c-48a8-b32e-0f291997d4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:46.278865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaf9bf-a22c-48a8-b32e-0f291997d4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:46.279137 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaf9bf-a22c-48a8-b32e-0f291997d4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:46.279413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eaf9bf-a22c-48a8-b32e-0f291997d4b3 None admin] PATH_INFO: `/v3/users/097c8cf926504f67b633e26214e21e23` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:46.294037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaf9bf-a22c-48a8-b32e-0f291997d4b3 None admin] RBAC: Authorizing `identity:get_user(user_id=097c8cf926504f67b633e26214e21e23)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:46.295479 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eaf9bf-a22c-48a8-b32e-0f291997d4b3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:46.297122 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 599/1182] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:46 2022] GET /identity/v3/users/097c8cf926504f67b633e26214e21e23 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:46.303062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7f1f0-8e04-45c2-898e-c58ac1946a97 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:46.399464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7f1f0-8e04-45c2-898e-c58ac1946a97 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:46.432701 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042a371-c58d-424e-b3a0-b4fd1de815cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:46.433300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a371-c58d-424e-b3a0-b4fd1de815cd None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:46.433577 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a371-c58d-424e-b3a0-b4fd1de815cd None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:46.433852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a371-c58d-424e-b3a0-b4fd1de815cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:46.469416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:46.505644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042a371-c58d-424e-b3a0-b4fd1de815cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:46.507204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042a371-c58d-424e-b3a0-b4fd1de815cd None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:46.606270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 584/1183] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:46.612313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaf9bf-a22c-48a8-b32e-0f291997d4b3 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:46.708931 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eaf9bf-a22c-48a8-b32e-0f291997d4b3 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:46.741472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb27591-6c10-4196-ab10-1c32e33908a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:46.742081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb27591-6c10-4196-ab10-1c32e33908a1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:46.742357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb27591-6c10-4196-ab10-1c32e33908a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:46.742624 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb27591-6c10-4196-ab10-1c32e33908a1 None admin] PATH_INFO: `/v3/projects/e4e85c90f2784488a4fd3531dbd3af54` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:46.743735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb27591-6c10-4196-ab10-1c32e33908a1 None admin] RBAC: Authorizing `identity:get_project(project_id=e4e85c90f2784488a4fd3531dbd3af54)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:46.745158 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb27591-6c10-4196-ab10-1c32e33908a1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:46.746768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 600/1184] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:46 2022] GET /identity/v3/projects/e4e85c90f2784488a4fd3531dbd3af54 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:46.753000 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042a371-c58d-424e-b3a0-b4fd1de815cd None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:46.812527 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:46.852549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042a371-c58d-424e-b3a0-b4fd1de815cd None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:46.887151 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff65113-9ccf-4bfc-9d04-8608867c1e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:46.887904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff65113-9ccf-4bfc-9d04-8608867c1e11 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:46.888188 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff65113-9ccf-4bfc-9d04-8608867c1e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:46.888458 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff65113-9ccf-4bfc-9d04-8608867c1e11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:46.889182 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff65113-9ccf-4bfc-9d04-8608867c1e11 None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=e4e85c90f2784488a4fd3531dbd3af54, user.id=097c8cf926504f67b633e26214e21e23) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:46.889489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff65113-9ccf-4bfc-9d04-8608867c1e11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:46.890908 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff65113-9ccf-4bfc-9d04-8608867c1e11 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:46.896362 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 585/1185] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:46 2022] GET /identity/v3/role_assignments?user.id=097c8cf926504f67b633e26214e21e23&scope.project.id=e4e85c90f2784488a4fd3531dbd3af54&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:48.764771 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac20da7-de7e-4ec7-8c5e-929d5ce2ef37 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:48.765306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac20da7-de7e-4ec7-8c5e-929d5ce2ef37 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:48.765306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac20da7-de7e-4ec7-8c5e-929d5ce2ef37 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:48.766659 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 601/1186] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:48.771720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499661a5-a122-4cd0-8c9f-9b8243d332b4 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:48.772112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499661a5-a122-4cd0-8c9f-9b8243d332b4 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:48.772397 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499661a5-a122-4cd0-8c9f-9b8243d332b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:48.790536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-499661a5-a122-4cd0-8c9f-9b8243d332b4 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:48.791757 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-499661a5-a122-4cd0-8c9f-9b8243d332b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:48.857467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 586/1187] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:49.125494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac20da7-de7e-4ec7-8c5e-929d5ce2ef37 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:49.172284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:49.226659 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac20da7-de7e-4ec7-8c5e-929d5ce2ef37 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:49.259322 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe54a51-1fca-48ff-96dd-298891bb5874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:49.259970 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe54a51-1fca-48ff-96dd-298891bb5874 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:49.260254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe54a51-1fca-48ff-96dd-298891bb5874 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:49.260527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe54a51-1fca-48ff-96dd-298891bb5874 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:49.270326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe54a51-1fca-48ff-96dd-298891bb5874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:49.275795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-2fe54a51-1fca-48ff-96dd-298891bb5874 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:49.275795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:49.275795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:49.275795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:49.275795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:49.275795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:49.275795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:49.275795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:49.275795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:49.275795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:49.275795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:49.275795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:49.275795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:49.275795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:49.275795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:49.275795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:49.276895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:49.276895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:49.276895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:49.276895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:49.276895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:49.276895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:49.276895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:49.276895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:49.276895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:49.276895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:49.276895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:49.276895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:49.276895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:49.276895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:49.276895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:49.276895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:49.277776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:49.277776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:49.277776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:49.277776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:49.277776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:49.277776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:49.277776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:49.277776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:49.277776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:49.277776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:49.277776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:49.277776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:21:49.277776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 602/1188] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:21:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:49.282899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499661a5-a122-4cd0-8c9f-9b8243d332b4 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:49.337736 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:49.383426 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499661a5-a122-4cd0-8c9f-9b8243d332b4 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:49.417621 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837db04-8cb2-4832-b422-5009a4c9dfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:49.418269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837db04-8cb2-4832-b422-5009a4c9dfa4 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:49.418602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837db04-8cb2-4832-b422-5009a4c9dfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:49.418924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d837db04-8cb2-4832-b422-5009a4c9dfa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:49.419640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837db04-8cb2-4832-b422-5009a4c9dfa4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:49.420015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837db04-8cb2-4832-b422-5009a4c9dfa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:49.421422 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d837db04-8cb2-4832-b422-5009a4c9dfa4 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:49.429341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 587/1189] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:21:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:49.435105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe54a51-1fca-48ff-96dd-298891bb5874 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:49.533110 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe54a51-1fca-48ff-96dd-298891bb5874 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:49.540948 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:49.566989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b8da0-794a-495e-9d97-e76b6a19aa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:49.567599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b8da0-794a-495e-9d97-e76b6a19aa84 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:49.567905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b8da0-794a-495e-9d97-e76b6a19aa84 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:49.568276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b8da0-794a-495e-9d97-e76b6a19aa84 None admin] PATH_INFO: `/v3/users/097c8cf926504f67b633e26214e21e23` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:49.570059 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b8da0-794a-495e-9d97-e76b6a19aa84 None admin] RBAC: Authorizing `identity:get_user(user_id=097c8cf926504f67b633e26214e21e23)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:49.571480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b8da0-794a-495e-9d97-e76b6a19aa84 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:49.573082 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 603/1190] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:49 2022] GET /identity/v3/users/097c8cf926504f67b633e26214e21e23 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:49.578927 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837db04-8cb2-4832-b422-5009a4c9dfa4 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:49.677704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d837db04-8cb2-4832-b422-5009a4c9dfa4 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:49.693877 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:49.711957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654447cd-6076-4bcc-9734-455591b5ddac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:49.712577 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654447cd-6076-4bcc-9734-455591b5ddac None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:49.712858 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654447cd-6076-4bcc-9734-455591b5ddac None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:49.713132 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654447cd-6076-4bcc-9734-455591b5ddac None admin] PATH_INFO: `/v3/projects/e4e85c90f2784488a4fd3531dbd3af54` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:49.714234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654447cd-6076-4bcc-9734-455591b5ddac None admin] RBAC: Authorizing `identity:get_project(project_id=e4e85c90f2784488a4fd3531dbd3af54)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:49.715691 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654447cd-6076-4bcc-9734-455591b5ddac None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:49.717326 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 588/1191] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:49 2022] GET /identity/v3/projects/e4e85c90f2784488a4fd3531dbd3af54 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:49.723283 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b8da0-794a-495e-9d97-e76b6a19aa84 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:49.818640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b8da0-794a-495e-9d97-e76b6a19aa84 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:49.853768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e84c14-f4f6-4e69-9c51-441ec835150a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:49.854429 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e84c14-f4f6-4e69-9c51-441ec835150a None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:49.854713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e84c14-f4f6-4e69-9c51-441ec835150a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:49.854989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e84c14-f4f6-4e69-9c51-441ec835150a None admin] PATH_INFO: `/v3/projects/e4e85c90f2784488a4fd3531dbd3af54/users/097c8cf926504f67b633e26214e21e23/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:49.856911 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e84c14-f4f6-4e69-9c51-441ec835150a None admin] RBAC: Authorizing `identity:create_grant(project_id=e4e85c90f2784488a4fd3531dbd3af54, user_id=097c8cf926504f67b633e26214e21e23, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:49.858345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e84c14-f4f6-4e69-9c51-441ec835150a None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:49.871452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 604/1192] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:21:49 2022] PUT /identity/v3/projects/e4e85c90f2784488a4fd3531dbd3af54/users/097c8cf926504f67b633e26214e21e23/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:21:51.768595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe468a0-d790-4d32-b31f-832959fb0820 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:51.769157 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe468a0-d790-4d32-b31f-832959fb0820 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:51.769157 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe468a0-d790-4d32-b31f-832959fb0820 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:51.770507 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 589/1193] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:21:51.775809 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6840e0-675d-4de2-b3d5-556b79ec0fce None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:51.776213 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6840e0-675d-4de2-b3d5-556b79ec0fce None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:51.776362 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6840e0-675d-4de2-b3d5-556b79ec0fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:51.794771 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ed6840e0-675d-4de2-b3d5-556b79ec0fce None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:21:51.796052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ed6840e0-675d-4de2-b3d5-556b79ec0fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:21:51.822252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:51.863198 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 605/1194] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:21:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:21:52.130256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe468a0-d790-4d32-b31f-832959fb0820 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:52.230827 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe468a0-d790-4d32-b31f-832959fb0820 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:52.263003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced7153-d377-468e-a3c5-e49b764cedde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:52.263648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced7153-d377-468e-a3c5-e49b764cedde None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:52.263931 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced7153-d377-468e-a3c5-e49b764cedde None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:52.264205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced7153-d377-468e-a3c5-e49b764cedde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:52.274640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced7153-d377-468e-a3c5-e49b764cedde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:21:52.279376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-7ced7153-d377-468e-a3c5-e49b764cedde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:52.279376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:21:52.279376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:21:52.279376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:21:52.279376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:21:52.279376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:21:52.279376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:21:52.279376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:21:52.279376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:21:52.279376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:21:52.279376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:21:52.279376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:21:52.279376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:21:52.279376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:21:52.279376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:21:52.279376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:21:52.280524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:21:52.280524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:21:52.280524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:21:52.280524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:21:52.280524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:21:52.280524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:21:52.280524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:21:52.280524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:21:52.280524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:21:52.280524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:21:52.280524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:21:52.280524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:21:52.280524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:21:52.280524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:21:52.280524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:21:52.280524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:21:52.281671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:21:52.281671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:21:52.281671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:21:52.281671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:21:52.281671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:21:52.281671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:21:52.281671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:21:52.281671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:21:52.281671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:21:52.281671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:21:52.281671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:21:52.281671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:21:52.281671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 590/1195] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:21:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:21:52.286477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6840e0-675d-4de2-b3d5-556b79ec0fce None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:52.382726 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6840e0-675d-4de2-b3d5-556b79ec0fce None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:52.415712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c2d53-8411-4a2f-af7a-686eb5a79ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:52.416303 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c2d53-8411-4a2f-af7a-686eb5a79ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:52.416577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c2d53-8411-4a2f-af7a-686eb5a79ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:52.416844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c2d53-8411-4a2f-af7a-686eb5a79ff1 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:52.417492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c2d53-8411-4a2f-af7a-686eb5a79ff1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:52.417787 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c2d53-8411-4a2f-af7a-686eb5a79ff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:52.419163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c2d53-8411-4a2f-af7a-686eb5a79ff1 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:52.426870 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 606/1196] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:21:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:52.432664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced7153-d377-468e-a3c5-e49b764cedde None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:52.484943 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:52.536426 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced7153-d377-468e-a3c5-e49b764cedde None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:52.570384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff7e5c-f4b0-405a-8f27-7b0e318b353a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:52.571008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff7e5c-f4b0-405a-8f27-7b0e318b353a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:52.571289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff7e5c-f4b0-405a-8f27-7b0e318b353a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:52.571571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff7e5c-f4b0-405a-8f27-7b0e318b353a None admin] PATH_INFO: `/v3/users/097c8cf926504f67b633e26214e21e23` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:52.572699 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff7e5c-f4b0-405a-8f27-7b0e318b353a None admin] RBAC: Authorizing `identity:get_user(user_id=097c8cf926504f67b633e26214e21e23)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:52.574113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff7e5c-f4b0-405a-8f27-7b0e318b353a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:52.575990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 591/1197] 10.222.0.58 () {56 vars in 1177 bytes} [Wed Aug 24 12:21:52 2022] GET /identity/v3/users/097c8cf926504f67b633e26214e21e23 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:21:52.581906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c2d53-8411-4a2f-af7a-686eb5a79ff1 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:52.609758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:52.680804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c2d53-8411-4a2f-af7a-686eb5a79ff1 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:52.714202 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead472a-c0e1-42b5-8fa9-71eb1d6022ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:52.714792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead472a-c0e1-42b5-8fa9-71eb1d6022ea None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:52.715077 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead472a-c0e1-42b5-8fa9-71eb1d6022ea None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:52.715357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead472a-c0e1-42b5-8fa9-71eb1d6022ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:52.780103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead472a-c0e1-42b5-8fa9-71eb1d6022ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:52.781549 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead472a-c0e1-42b5-8fa9-71eb1d6022ea None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:52.807753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:52.881866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 607/1198] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:21:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:21:52.887822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff7e5c-f4b0-405a-8f27-7b0e318b353a None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:52.986092 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff7e5c-f4b0-405a-8f27-7b0e318b353a None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:52.999286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:53.018743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dcd7f-bfb4-489a-8583-921e70a0c091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:53.019355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dcd7f-bfb4-489a-8583-921e70a0c091 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:53.019641 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dcd7f-bfb4-489a-8583-921e70a0c091 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:53.019924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dcd7f-bfb4-489a-8583-921e70a0c091 None admin] PATH_INFO: `/v3/projects/e4e85c90f2784488a4fd3531dbd3af54` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:53.021006 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988dcd7f-bfb4-489a-8583-921e70a0c091 None admin] RBAC: Authorizing `identity:get_project(project_id=e4e85c90f2784488a4fd3531dbd3af54)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:53.023011 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988dcd7f-bfb4-489a-8583-921e70a0c091 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:53.024689 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 592/1199] 10.222.0.58 () {56 vars in 1186 bytes} [Wed Aug 24 12:21:52 2022] GET /identity/v3/projects/e4e85c90f2784488a4fd3531dbd3af54 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:21:53.030828 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead472a-c0e1-42b5-8fa9-71eb1d6022ea None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:21:53.125852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead472a-c0e1-42b5-8fa9-71eb1d6022ea None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:21:53.150833 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:21:53.159871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655147f3-0377-4e5e-b8f1-1d78851bbb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:21:53.160472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655147f3-0377-4e5e-b8f1-1d78851bbb47 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:21:53.160756 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655147f3-0377-4e5e-b8f1-1d78851bbb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:21:53.161036 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655147f3-0377-4e5e-b8f1-1d78851bbb47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:21:53.161762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655147f3-0377-4e5e-b8f1-1d78851bbb47 None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=e4e85c90f2784488a4fd3531dbd3af54, user.id=097c8cf926504f67b633e26214e21e23) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:21:53.162073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655147f3-0377-4e5e-b8f1-1d78851bbb47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:21:53.164094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655147f3-0377-4e5e-b8f1-1d78851bbb47 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:21:53.169722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 608/1200] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:21:53 2022] GET /identity/v3/role_assignments?user.id=097c8cf926504f67b633e26214e21e23&scope.project.id=e4e85c90f2784488a4fd3531dbd3af54&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:22:03.015241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=9 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:03.167257 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:49.110189 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fcfba-f40a-49f5-a2b2-871065e80f42 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:49.110189 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fcfba-f40a-49f5-a2b2-871065e80f42 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:49.110189 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fcfba-f40a-49f5-a2b2-871065e80f42 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:49.111147 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 593/1201] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:22:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:49.116535 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869544d8-45b8-48e3-a461-eb03af594e4a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:49.117063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869544d8-45b8-48e3-a461-eb03af594e4a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:49.117149 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869544d8-45b8-48e3-a461-eb03af594e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:49.138067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-869544d8-45b8-48e3-a461-eb03af594e4a None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:49.139854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-869544d8-45b8-48e3-a461-eb03af594e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:49.230872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 609/1202] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:22:49.505819 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63fcfba-f40a-49f5-a2b2-871065e80f42 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:49.643953 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63fcfba-f40a-49f5-a2b2-871065e80f42 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:49.689045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f8531-41bf-42c9-80cb-1e579aa86996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:49.689689 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8531-41bf-42c9-80cb-1e579aa86996 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:49.689971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8531-41bf-42c9-80cb-1e579aa86996 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:49.690243 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8531-41bf-42c9-80cb-1e579aa86996 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:49.701607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f8531-41bf-42c9-80cb-1e579aa86996 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:49.703068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f8531-41bf-42c9-80cb-1e579aa86996 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:49.721637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-394f8531-41bf-42c9-80cb-1e579aa86996 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:22:49.721637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:22:49.721637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:22:49.721637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:22:49.721637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:22:49.721637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:22:49.721637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:22:49.721637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:22:49.721637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:22:49.721637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:22:49.721637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:22:49.721637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:22:49.721637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:22:49.721637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:22:49.721637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:22:49.721637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:22:49.723817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:22:49.725341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:22:49.725341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:22:49.725341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:22:49.725341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 594/1203] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:22:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:22:49.730395 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869544d8-45b8-48e3-a461-eb03af594e4a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:49.863321 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869544d8-45b8-48e3-a461-eb03af594e4a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:49.905148 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:49.914143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7083e0a-7f67-4864-a96a-baa24e567425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:49.915108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7083e0a-7f67-4864-a96a-baa24e567425 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:49.915516 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7083e0a-7f67-4864-a96a-baa24e567425 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:49.915916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7083e0a-7f67-4864-a96a-baa24e567425 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:49.916674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7083e0a-7f67-4864-a96a-baa24e567425 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:22:49.917099 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7083e0a-7f67-4864-a96a-baa24e567425 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:49.918572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7083e0a-7f67-4864-a96a-baa24e567425 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:49.936217 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 610/1204] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:22:49.938256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f8531-41bf-42c9-80cb-1e579aa86996 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:50.011221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:50.083940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f8531-41bf-42c9-80cb-1e579aa86996 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:50.132206 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ba831-7c1c-4e0e-9e92-e6a483b398e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:50.132840 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ba831-7c1c-4e0e-9e92-e6a483b398e8 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:50.133042 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ba831-7c1c-4e0e-9e92-e6a483b398e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:50.133269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ba831-7c1c-4e0e-9e92-e6a483b398e8 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:50.133950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ba831-7c1c-4e0e-9e92-e6a483b398e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:50.135397 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ba831-7c1c-4e0e-9e92-e6a483b398e8 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:50.175875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 595/1205] 10.222.0.58 () {60 vars in 1132 bytes} [Wed Aug 24 12:22:49 2022] POST /identity/v3/users => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:22:52.322500 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f3438-48d1-4feb-82cb-ae2f63d8b5c1 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:52.323051 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f3438-48d1-4feb-82cb-ae2f63d8b5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:52.323144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f3438-48d1-4feb-82cb-ae2f63d8b5c1 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:52.325005 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 611/1206] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:22:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:52.330473 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e79ce1-f810-4682-a155-7db192fc9068 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:52.330896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e79ce1-f810-4682-a155-7db192fc9068 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:52.331329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e79ce1-f810-4682-a155-7db192fc9068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:52.352277 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-72e79ce1-f810-4682-a155-7db192fc9068 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:52.353831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-72e79ce1-f810-4682-a155-7db192fc9068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:52.433077 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 596/1207] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:22:52.751423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f3438-48d1-4feb-82cb-ae2f63d8b5c1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:52.887539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f3438-48d1-4feb-82cb-ae2f63d8b5c1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:52.927805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36733372-5660-40d0-928d-94e5a7149f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:52.928467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36733372-5660-40d0-928d-94e5a7149f3c None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:52.928750 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36733372-5660-40d0-928d-94e5a7149f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:52.929019 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36733372-5660-40d0-928d-94e5a7149f3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:52.940850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36733372-5660-40d0-928d-94e5a7149f3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:22:52.947499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-36733372-5660-40d0-928d-94e5a7149f3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:22:52.947499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:22:52.947499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:22:52.947499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:22:52.947499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:22:52.947499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:22:52.947499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:22:52.947499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:22:52.947499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:22:52.947499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:22:52.947499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:22:52.947499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:22:52.947499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:22:52.947499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:22:52.947499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:22:52.947499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:22:52.948871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:22:52.948871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:22:52.948871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:22:52.948871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:22:52.948871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:22:52.948871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:22:52.948871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:22:52.948871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:22:52.948871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:22:52.948871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:22:52.948871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:22:52.948871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:22:52.948871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:22:52.948871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:22:52.948871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:22:52.948871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:22:52.949758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:22:52.949758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:22:52.949758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:22:52.949758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:22:52.949758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:22:52.949758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:22:52.949758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:22:52.949758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:22:52.949758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:22:52.949758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:22:52.949758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:22:52.949758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:22:52.949758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 612/1208] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:22:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:22:52.956719 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e79ce1-f810-4682-a155-7db192fc9068 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:52.998180 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:53.084655 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e79ce1-f810-4682-a155-7db192fc9068 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:53.131230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43631411-32ef-4931-bec0-3d327552fa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:53.131875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43631411-32ef-4931-bec0-3d327552fa9e None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:53.133015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43631411-32ef-4931-bec0-3d327552fa9e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:53.133015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43631411-32ef-4931-bec0-3d327552fa9e None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:53.133181 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43631411-32ef-4931-bec0-3d327552fa9e None admin] RBAC: Adding query filter params (name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:22:53.133379 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43631411-32ef-4931-bec0-3d327552fa9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:53.134830 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43631411-32ef-4931-bec0-3d327552fa9e None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:53.144024 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 597/1209] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:22:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:22:53.150001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36733372-5660-40d0-928d-94e5a7149f3c None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:53.213725 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:53.277609 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36733372-5660-40d0-928d-94e5a7149f3c None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:53.315159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f8df0-24e4-4620-b3c0-74c144b26a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:53.315996 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f8df0-24e4-4620-b3c0-74c144b26a79 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:53.316277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f8df0-24e4-4620-b3c0-74c144b26a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:53.316629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f8df0-24e4-4620-b3c0-74c144b26a79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:53.328732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f8df0-24e4-4620-b3c0-74c144b26a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:53.330232 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f8df0-24e4-4620-b3c0-74c144b26a79 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:53.346167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-b79f8df0-24e4-4620-b3c0-74c144b26a79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:22:53.346167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:22:53.346167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:22:53.346167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:22:53.346167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:22:53.346167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:22:53.346167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:22:53.346167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:22:53.346167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:22:53.346167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:22:53.346167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:22:53.346167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:22:53.346167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:22:53.346167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:22:53.346167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:22:53.346167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:22:53.347349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:22:53.349428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:22:53.349428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:22:53.349428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:22:53.350934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 613/1210] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:22:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:22:53.360086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43631411-32ef-4931-bec0-3d327552fa9e None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:53.470682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:53.475050 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43631411-32ef-4931-bec0-3d327552fa9e None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:53.513403 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54f19c-363c-45af-9113-ce31e3e18134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:53.514017 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54f19c-363c-45af-9113-ce31e3e18134 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:53.514318 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54f19c-363c-45af-9113-ce31e3e18134 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:53.514568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54f19c-363c-45af-9113-ce31e3e18134 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:53.515216 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54f19c-363c-45af-9113-ce31e3e18134 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:22:53.515519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54f19c-363c-45af-9113-ce31e3e18134 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:53.516931 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54f19c-363c-45af-9113-ce31e3e18134 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:53.529606 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 598/1211] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:22:53.535996 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f8df0-24e4-4620-b3c0-74c144b26a79 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:53.648649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f8df0-24e4-4620-b3c0-74c144b26a79 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:53.650204 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:53.688067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2f931-be62-4b0c-aca6-4ec12b80318b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:53.688738 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2f931-be62-4b0c-aca6-4ec12b80318b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:53.689109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2f931-be62-4b0c-aca6-4ec12b80318b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:53.689389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2f931-be62-4b0c-aca6-4ec12b80318b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:53.698300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2f931-be62-4b0c-aca6-4ec12b80318b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:53.700072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2f931-be62-4b0c-aca6-4ec12b80318b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:53.711723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-10f2f931-be62-4b0c-aca6-4ec12b80318b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:22:53.711723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:22:53.711723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:22:53.711723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:22:53.711723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:22:53.711723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:22:53.711723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:22:53.711723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:22:53.711723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:22:53.711723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:22:53.711723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:22:53.711723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:22:53.711723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:22:53.711723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:22:53.711723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:22:53.711723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:22:53.713398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:22:53.714384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:22:53.714384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:22:53.714384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:22:53.714384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:22:53.714384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:22:53.714384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:22:53.714384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:22:53.714384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:22:53.714384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:22:53.714384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:22:53.714384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 614/1212] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:22:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:22:53.718645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54f19c-363c-45af-9113-ce31e3e18134 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:53.834594 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54f19c-363c-45af-9113-ce31e3e18134 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:53.871596 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaef276-0dbe-40da-9d0f-582618550ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:53.872382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaef276-0dbe-40da-9d0f-582618550ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:53.872761 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaef276-0dbe-40da-9d0f-582618550ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:53.874215 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaef276-0dbe-40da-9d0f-582618550ea9 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:53.874215 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaef276-0dbe-40da-9d0f-582618550ea9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:22:53.874215 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaef276-0dbe-40da-9d0f-582618550ea9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:53.875918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaef276-0dbe-40da-9d0f-582618550ea9 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:53.898123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 599/1213] 10.222.0.58 () {56 vars in 1143 bytes} [Wed Aug 24 12:22:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:22:53.904477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2f931-be62-4b0c-aca6-4ec12b80318b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:54.016802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2f931-be62-4b0c-aca6-4ec12b80318b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:54.056196 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4838b-a8b1-4951-92be-e924151936d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:54.056864 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4838b-a8b1-4951-92be-e924151936d8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:54.057148 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4838b-a8b1-4951-92be-e924151936d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:54.057426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4838b-a8b1-4951-92be-e924151936d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:54.095567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:54.141519 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4838b-a8b1-4951-92be-e924151936d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:54.143046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4838b-a8b1-4951-92be-e924151936d8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:54.269076 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 615/1214] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:22:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:22:54.275186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaef276-0dbe-40da-9d0f-582618550ea9 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:54.302610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:54.389723 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaef276-0dbe-40da-9d0f-582618550ea9 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:54.429496 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d004c-3d01-4e3a-96bf-462904d176bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:54.430142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d004c-3d01-4e3a-96bf-462904d176bb None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:54.430423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d004c-3d01-4e3a-96bf-462904d176bb None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:54.430697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9d004c-3d01-4e3a-96bf-462904d176bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:54.442037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d004c-3d01-4e3a-96bf-462904d176bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:54.443509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9d004c-3d01-4e3a-96bf-462904d176bb None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:54.459292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-6d9d004c-3d01-4e3a-96bf-462904d176bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:22:54.459292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:22:54.459292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:22:54.459292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:22:54.459292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:22:54.459292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:22:54.459292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:22:54.459292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:22:54.459292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:22:54.459292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:22:54.459292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:22:54.459292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:22:54.459292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:22:54.459292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:22:54.459292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:22:54.459292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:22:54.460525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:22:54.461461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:22:54.461461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:22:54.461461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:22:54.461461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 600/1215] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:22:54.466548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4838b-a8b1-4951-92be-e924151936d8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:54.537171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:54.581337 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4838b-a8b1-4951-92be-e924151936d8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:54.617139 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9718e2a-37f6-40b0-8861-15c5ad2f96d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:54.617847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9718e2a-37f6-40b0-8861-15c5ad2f96d6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:54.618131 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9718e2a-37f6-40b0-8861-15c5ad2f96d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:54.618406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9718e2a-37f6-40b0-8861-15c5ad2f96d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:54.619057 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9718e2a-37f6-40b0-8861-15c5ad2f96d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:22:54.619362 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9718e2a-37f6-40b0-8861-15c5ad2f96d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:54.620770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9718e2a-37f6-40b0-8861-15c5ad2f96d6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:54.637814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 616/1216] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:22:54.639111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d004c-3d01-4e3a-96bf-462904d176bb None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:54.714315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:54.748562 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9d004c-3d01-4e3a-96bf-462904d176bb None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:54.784895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe617074-1787-475e-a282-a60826df8f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:54.786134 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe617074-1787-475e-a282-a60826df8f94 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:54.786441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe617074-1787-475e-a282-a60826df8f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:54.786693 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe617074-1787-475e-a282-a60826df8f94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:54.795979 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe617074-1787-475e-a282-a60826df8f94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:54.797362 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe617074-1787-475e-a282-a60826df8f94 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:54.809447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-fe617074-1787-475e-a282-a60826df8f94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:22:54.809447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:22:54.809447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:22:54.809447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:22:54.809447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:22:54.809447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:22:54.809447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:22:54.809447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:22:54.809447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:22:54.809447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:22:54.809447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:22:54.809447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:22:54.809447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:22:54.809447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:22:54.809447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:22:54.810399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:22:54.811398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:22:54.811398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:22:54.811398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:22:54.811398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:22:54.811398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:22:54.811398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:22:54.811398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:22:54.811398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:22:54.811398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 601/1217] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:22:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:22:54.816617 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9718e2a-37f6-40b0-8861-15c5ad2f96d6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:54.935360 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9718e2a-37f6-40b0-8861-15c5ad2f96d6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:54.965087 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:54.978864 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad1040-c994-46fa-9850-9b7785fd5c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:54.979460 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad1040-c994-46fa-9850-9b7785fd5c98 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:54.979825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad1040-c994-46fa-9850-9b7785fd5c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:54.980061 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad1040-c994-46fa-9850-9b7785fd5c98 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:54.980777 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad1040-c994-46fa-9850-9b7785fd5c98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:22:54.981079 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad1040-c994-46fa-9850-9b7785fd5c98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:54.982612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad1040-c994-46fa-9850-9b7785fd5c98 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:55.005174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 617/1218] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:22:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:22:55.006216 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe617074-1787-475e-a282-a60826df8f94 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:55.125500 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe617074-1787-475e-a282-a60826df8f94 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:55.156021 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:55.163446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0e75b-655b-4278-90d9-252d9dc80d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:55.164404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0e75b-655b-4278-90d9-252d9dc80d34 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:55.164913 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0e75b-655b-4278-90d9-252d9dc80d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:55.165417 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0e75b-655b-4278-90d9-252d9dc80d34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:55.166382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0e75b-655b-4278-90d9-252d9dc80d34 None admin] RBAC: Adding query filter params (role.id=5a12926bd458420e81e72ecf825d67ae, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=b7ba45e3cff74962965b248f9f017f11) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:22:55.166857 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0e75b-655b-4278-90d9-252d9dc80d34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:55.168752 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0e75b-655b-4278-90d9-252d9dc80d34 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:55.182560 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 602/1219] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:22:55 2022] GET /identity/v3/role_assignments?user.id=b7ba45e3cff74962965b248f9f017f11&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=5a12926bd458420e81e72ecf825d67ae => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:22:57.381838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f29dc-01ed-45a3-90c2-ab1ee7e4491f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:57.382430 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f29dc-01ed-45a3-90c2-ab1ee7e4491f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:57.382430 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f29dc-01ed-45a3-90c2-ab1ee7e4491f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:57.383800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 618/1220] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:22:57.390211 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9cd8d-68aa-4c11-90c8-6210045f79b1 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:57.390881 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9cd8d-68aa-4c11-90c8-6210045f79b1 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:57.391025 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9cd8d-68aa-4c11-90c8-6210045f79b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:57.413311 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-2fc9cd8d-68aa-4c11-90c8-6210045f79b1 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:22:57.414620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2fc9cd8d-68aa-4c11-90c8-6210045f79b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:22:57.492657 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 603/1221] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:22:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 12:22:57.790797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f29dc-01ed-45a3-90c2-ab1ee7e4491f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:57.906467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f29dc-01ed-45a3-90c2-ab1ee7e4491f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:57.943902 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed9567-2609-441b-a1ac-6145a390a0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:57.944530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed9567-2609-441b-a1ac-6145a390a0df None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:57.944815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed9567-2609-441b-a1ac-6145a390a0df None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:57.945076 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed9567-2609-441b-a1ac-6145a390a0df None admin] PATH_INFO: `/v3/roles/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:57.956827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aed9567-2609-441b-a1ac-6145a390a0df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:22:57.962568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-6aed9567-2609-441b-a1ac-6145a390a0df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:22:57.962568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:22:57.962568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:22:57.962568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:22:57.962568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:22:57.962568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:22:57.962568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:22:57.962568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:22:57.962568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:22:57.962568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:22:57.962568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:22:57.962568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:22:57.962568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:22:57.962568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:22:57.962568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:22:57.962568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:22:57.963761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:22:57.963761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:22:57.963761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:22:57.963761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:22:57.963761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:22:57.963761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:22:57.963761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:22:57.963761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:22:57.963761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:22:57.963761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:22:57.963761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:22:57.963761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:22:57.963761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:22:57.963761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:22:57.963761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:22:57.963761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:22:57.964644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:22:57.964644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:22:57.964644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:22:57.964644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:22:57.964644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:22:57.964644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:22:57.964644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:22:57.964644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:22:57.964644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:22:57.964644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:22:57.964644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:22:57.964644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:22:57.964644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 619/1222] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:22:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:22:57.970486 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9cd8d-68aa-4c11-90c8-6210045f79b1 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:58.079257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9cd8d-68aa-4c11-90c8-6210045f79b1 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:58.105868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:58.118704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935df2d-876c-427c-b7fe-5199bd16d2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:58.119376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935df2d-876c-427c-b7fe-5199bd16d2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:58.119680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935df2d-876c-427c-b7fe-5199bd16d2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:58.119955 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a935df2d-876c-427c-b7fe-5199bd16d2b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:58.120620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935df2d-876c-427c-b7fe-5199bd16d2b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:22:58.120923 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935df2d-876c-427c-b7fe-5199bd16d2b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:58.122284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a935df2d-876c-427c-b7fe-5199bd16d2b2 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:58.131003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 604/1223] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:22:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:22:58.137009 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed9567-2609-441b-a1ac-6145a390a0df None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:58.186546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:58.257409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed9567-2609-441b-a1ac-6145a390a0df None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:58.295528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd852f6-2b8b-42d2-a64e-34dd2d5f35ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:58.296213 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd852f6-2b8b-42d2-a64e-34dd2d5f35ab None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:58.296492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd852f6-2b8b-42d2-a64e-34dd2d5f35ab None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:58.296770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd852f6-2b8b-42d2-a64e-34dd2d5f35ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:58.307713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd852f6-2b8b-42d2-a64e-34dd2d5f35ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:58.309168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd852f6-2b8b-42d2-a64e-34dd2d5f35ab None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:58.324739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-2dd852f6-2b8b-42d2-a64e-34dd2d5f35ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:22:58.324739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:22:58.324739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:22:58.324739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:22:58.324739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:22:58.324739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:22:58.324739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:22:58.324739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:22:58.324739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:22:58.324739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:22:58.324739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:22:58.324739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:22:58.324739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:22:58.324739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:22:58.324739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:22:58.324739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:22:58.325999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:22:58.327089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:22:58.327089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:22:58.327089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:22:58.327089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 620/1224] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:22:58.331966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935df2d-876c-427c-b7fe-5199bd16d2b2 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:58.437226 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a935df2d-876c-427c-b7fe-5199bd16d2b2 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:58.476476 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be22094-6345-49b8-ae46-ff2cd878b888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:58.477193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be22094-6345-49b8-ae46-ff2cd878b888 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:58.477577 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be22094-6345-49b8-ae46-ff2cd878b888 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:58.477922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be22094-6345-49b8-ae46-ff2cd878b888 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:58.479162 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be22094-6345-49b8-ae46-ff2cd878b888 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:22:58.479469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be22094-6345-49b8-ae46-ff2cd878b888 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:58.480978 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be22094-6345-49b8-ae46-ff2cd878b888 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:58.494250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 605/1225] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:22:58.500564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd852f6-2b8b-42d2-a64e-34dd2d5f35ab None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:58.604947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:58.614174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd852f6-2b8b-42d2-a64e-34dd2d5f35ab None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:58.653872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253ab9d-9bac-41eb-9dc4-456c5f6bd4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:58.654731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253ab9d-9bac-41eb-9dc4-456c5f6bd4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:58.655034 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253ab9d-9bac-41eb-9dc4-456c5f6bd4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:58.655425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253ab9d-9bac-41eb-9dc4-456c5f6bd4c7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:58.663615 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253ab9d-9bac-41eb-9dc4-456c5f6bd4c7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:58.665342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253ab9d-9bac-41eb-9dc4-456c5f6bd4c7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:58.678207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-f253ab9d-9bac-41eb-9dc4-456c5f6bd4c7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:22:58.678207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:22:58.678207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:22:58.678207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:22:58.678207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:22:58.678207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:22:58.678207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:22:58.678207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:22:58.678207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:22:58.678207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:22:58.678207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:22:58.678207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:22:58.678207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:22:58.678207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:22:58.678207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:22:58.678207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:22:58.679492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:22:58.680824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:22:58.680824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:22:58.680824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:22:58.680824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:22:58.680824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:22:58.680824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:22:58.680824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:22:58.680824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:22:58.680824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:22:58.680824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:22:58.680824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 621/1226] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:22:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:22:58.685703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be22094-6345-49b8-ae46-ff2cd878b888 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:58.744168 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:58.804726 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be22094-6345-49b8-ae46-ff2cd878b888 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:58.841193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004bfbc-cf37-4603-89bc-ee9642e8b6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:58.841789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004bfbc-cf37-4603-89bc-ee9642e8b6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:58.842070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004bfbc-cf37-4603-89bc-ee9642e8b6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:58.842428 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004bfbc-cf37-4603-89bc-ee9642e8b6e9 None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:58.843076 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004bfbc-cf37-4603-89bc-ee9642e8b6e9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:22:58.843385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004bfbc-cf37-4603-89bc-ee9642e8b6e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:58.844865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004bfbc-cf37-4603-89bc-ee9642e8b6e9 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:58.870016 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 606/1227] 10.222.0.58 () {56 vars in 1143 bytes} [Wed Aug 24 12:22:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:22:58.871652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253ab9d-9bac-41eb-9dc4-456c5f6bd4c7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:58.992741 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253ab9d-9bac-41eb-9dc4-456c5f6bd4c7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:59.029858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f41bee-e09f-48ea-a58e-9a5905ab128b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:59.030490 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f41bee-e09f-48ea-a58e-9a5905ab128b None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:59.030773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f41bee-e09f-48ea-a58e-9a5905ab128b None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:59.031049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f41bee-e09f-48ea-a58e-9a5905ab128b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:59.043260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f41bee-e09f-48ea-a58e-9a5905ab128b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:59.044755 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f41bee-e09f-48ea-a58e-9a5905ab128b None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:59.059504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-97f41bee-e09f-48ea-a58e-9a5905ab128b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:22:59.059504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:22:59.059504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:22:59.059504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:22:59.059504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:22:59.059504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:22:59.059504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:22:59.059504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:22:59.059504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:22:59.059504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:22:59.059504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:22:59.059504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:22:59.059504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:22:59.059504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:22:59.059504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:22:59.059504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:22:59.060994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:22:59.062214 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:22:59.062214 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:22:59.062214 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:22:59.062214 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 622/1228] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:22:59.067045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004bfbc-cf37-4603-89bc-ee9642e8b6e9 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:59.182872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004bfbc-cf37-4603-89bc-ee9642e8b6e9 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:59.185175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:59.223517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788a9d9-b3a8-46c1-9ade-7481def56cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:59.224160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788a9d9-b3a8-46c1-9ade-7481def56cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:59.224441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788a9d9-b3a8-46c1-9ade-7481def56cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:59.224714 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788a9d9-b3a8-46c1-9ade-7481def56cf8 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:59.225350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788a9d9-b3a8-46c1-9ade-7481def56cf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:22:59.225651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788a9d9-b3a8-46c1-9ade-7481def56cf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:59.227027 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788a9d9-b3a8-46c1-9ade-7481def56cf8 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:59.239451 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 607/1229] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:22:59.245449 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f41bee-e09f-48ea-a58e-9a5905ab128b None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:59.255732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:59.357418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f41bee-e09f-48ea-a58e-9a5905ab128b None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:59.400160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be7244-d732-4a0e-8c4e-5a17003fb293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:59.400806 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be7244-d732-4a0e-8c4e-5a17003fb293 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:59.401094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be7244-d732-4a0e-8c4e-5a17003fb293 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:59.401371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be7244-d732-4a0e-8c4e-5a17003fb293 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:59.408317 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be7244-d732-4a0e-8c4e-5a17003fb293 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:59.409772 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be7244-d732-4a0e-8c4e-5a17003fb293 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:59.422702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-27be7244-d732-4a0e-8c4e-5a17003fb293 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:22:59.422702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:22:59.422702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:22:59.422702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:22:59.422702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:22:59.422702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:22:59.422702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:22:59.422702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:22:59.422702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:22:59.422702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:22:59.422702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:22:59.422702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:22:59.422702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:22:59.422702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:22:59.422702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:22:59.423805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:22:59.424749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:22:59.424749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:22:59.424749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:22:59.424749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:22:59.424749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:22:59.424749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:22:59.424749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:22:59.424749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:22:59.424749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 623/1230] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:22:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:22:59.430725 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788a9d9-b3a8-46c1-9ade-7481def56cf8 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:59.549027 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788a9d9-b3a8-46c1-9ade-7481def56cf8 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:59.588835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64519a38-2262-4d26-99f7-95967ffb558a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:59.589420 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64519a38-2262-4d26-99f7-95967ffb558a None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:59.589730 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64519a38-2262-4d26-99f7-95967ffb558a None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:59.589975 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64519a38-2262-4d26-99f7-95967ffb558a None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:59.590670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64519a38-2262-4d26-99f7-95967ffb558a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:22:59.590979 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64519a38-2262-4d26-99f7-95967ffb558a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:59.592447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64519a38-2262-4d26-99f7-95967ffb558a None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:59.605844 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 608/1231] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:22:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:22:59.612761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be7244-d732-4a0e-8c4e-5a17003fb293 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:22:59.689048 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:22:59.730075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be7244-d732-4a0e-8c4e-5a17003fb293 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:22:59.769289 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9310f-d8d5-4264-a5cb-8801ca62e648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:22:59.769958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9310f-d8d5-4264-a5cb-8801ca62e648 None admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:22:59.770233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9310f-d8d5-4264-a5cb-8801ca62e648 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:22:59.770497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9310f-d8d5-4264-a5cb-8801ca62e648 None admin] PATH_INFO: `/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/b7ba45e3cff74962965b248f9f017f11/roles/5a12926bd458420e81e72ecf825d67ae` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:22:59.785730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9310f-d8d5-4264-a5cb-8801ca62e648 None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user_id=b7ba45e3cff74962965b248f9f017f11, role_id=5a12926bd458420e81e72ecf825d67ae)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:22:59.787424 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9310f-d8d5-4264-a5cb-8801ca62e648 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:22:59.800265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 624/1232] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:22:59 2022] PUT /identity/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/b7ba45e3cff74962965b248f9f017f11/roles/5a12926bd458420e81e72ecf825d67ae => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:23:01.805241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1401a-0cc2-4c47-b28e-46d6656dbbb4 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:01.805799 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1401a-0cc2-4c47-b28e-46d6656dbbb4 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:01.805799 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f1401a-0cc2-4c47-b28e-46d6656dbbb4 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:01.807142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 609/1233] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:01.812361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a2037-cd98-4f09-929c-b8d10605cad8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:01.812759 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a2037-cd98-4f09-929c-b8d10605cad8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:01.812923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a2037-cd98-4f09-929c-b8d10605cad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:01.833147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-330a2037-cd98-4f09-929c-b8d10605cad8 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:01.834513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-330a2037-cd98-4f09-929c-b8d10605cad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:01.909642 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 625/1234] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:23:02.185969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1401a-0cc2-4c47-b28e-46d6656dbbb4 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:02.218831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:02.322595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f1401a-0cc2-4c47-b28e-46d6656dbbb4 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:02.362013 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de784ec-814b-4424-ada5-265b3ff470ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:02.363682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de784ec-814b-4424-ada5-265b3ff470ef None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:02.364142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de784ec-814b-4424-ada5-265b3ff470ef None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:02.364599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de784ec-814b-4424-ada5-265b3ff470ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:02.375858 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de784ec-814b-4424-ada5-265b3ff470ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:23:02.381109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-2de784ec-814b-4424-ada5-265b3ff470ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:23:02.381109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:02.381109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:02.381109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:02.381109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:02.381109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:02.381109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:02.381109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:02.381109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:02.381109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:02.381109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:02.381109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:02.381109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:23:02.381109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:23:02.381109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:23:02.381109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:23:02.382299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:23:02.382299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:23:02.382299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:23:02.382299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:23:02.382299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:02.382299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:02.382299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:02.382299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:02.382299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:02.382299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:02.382299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:02.382299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:02.382299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:02.382299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:02.382299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:02.382299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:02.383415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:02.383415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:02.383415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:02.383415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:02.383415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:23:02.383415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:23:02.383415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:23:02.383415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:23:02.383415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:23:02.383415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:23:02.383415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 12:23:02.383415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:23:02.387477 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 610/1235] 10.222.0.58 () {56 vars in 1102 bytes} [Wed Aug 24 12:23:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:02.389052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a2037-cd98-4f09-929c-b8d10605cad8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:02.439813 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:02.504363 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a2037-cd98-4f09-929c-b8d10605cad8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:02.544907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366a191-8c99-48b0-a2a2-614f02f5b952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:02.545537 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366a191-8c99-48b0-a2a2-614f02f5b952 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:02.545814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366a191-8c99-48b0-a2a2-614f02f5b952 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:02.546166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366a191-8c99-48b0-a2a2-614f02f5b952 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:02.546991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366a191-8c99-48b0-a2a2-614f02f5b952 None admin] RBAC: Adding query filter params (name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:02.547300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366a191-8c99-48b0-a2a2-614f02f5b952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:02.548873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366a191-8c99-48b0-a2a2-614f02f5b952 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:02.557836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 626/1236] 10.222.0.58 () {56 vars in 1103 bytes} [Wed Aug 24 12:23:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:23:02.563690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de784ec-814b-4424-ada5-265b3ff470ef None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:02.657707 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:02.677769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de784ec-814b-4424-ada5-265b3ff470ef None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:02.716463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da3a1e-2070-4e61-9fb0-aa2c4dcfa43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:02.717220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da3a1e-2070-4e61-9fb0-aa2c4dcfa43d None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:02.717625 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da3a1e-2070-4e61-9fb0-aa2c4dcfa43d None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:02.718022 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da3a1e-2070-4e61-9fb0-aa2c4dcfa43d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:02.731944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da3a1e-2070-4e61-9fb0-aa2c4dcfa43d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:02.733665 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da3a1e-2070-4e61-9fb0-aa2c4dcfa43d None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:02.751160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-44da3a1e-2070-4e61-9fb0-aa2c4dcfa43d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:02.751160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:02.751160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:02.751160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:02.751160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:02.751160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:02.751160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:02.751160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:02.751160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:02.751160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:02.751160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:02.751160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:02.751160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:02.751160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:02.751160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:02.751160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:02.752664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:02.754697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:02.754697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:02.754697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:23:02.754697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 611/1237] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:02.760721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366a191-8c99-48b0-a2a2-614f02f5b952 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:02.879536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366a191-8c99-48b0-a2a2-614f02f5b952 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:02.889956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:02.917860 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fd3c3-58a0-42ba-98de-0e92fd753397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:02.918671 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fd3c3-58a0-42ba-98de-0e92fd753397 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:02.919063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fd3c3-58a0-42ba-98de-0e92fd753397 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:02.919342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18fd3c3-58a0-42ba-98de-0e92fd753397 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:02.920248 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fd3c3-58a0-42ba-98de-0e92fd753397 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:02.920554 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fd3c3-58a0-42ba-98de-0e92fd753397 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:02.921916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18fd3c3-58a0-42ba-98de-0e92fd753397 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:02.936577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 627/1238] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:02.944443 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da3a1e-2070-4e61-9fb0-aa2c4dcfa43d None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:03.086388 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da3a1e-2070-4e61-9fb0-aa2c4dcfa43d None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:03.125553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570a7b8-a476-475e-bb93-a2ed0b73aa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:03.126305 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570a7b8-a476-475e-bb93-a2ed0b73aa3f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:03.126590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570a7b8-a476-475e-bb93-a2ed0b73aa3f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:03.126861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570a7b8-a476-475e-bb93-a2ed0b73aa3f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:03.131808 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:03.138069 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570a7b8-a476-475e-bb93-a2ed0b73aa3f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:03.139829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570a7b8-a476-475e-bb93-a2ed0b73aa3f None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:03.153850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-0570a7b8-a476-475e-bb93-a2ed0b73aa3f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:03.153850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:03.153850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:03.153850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:03.153850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:03.153850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:03.153850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:03.153850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:03.153850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:03.153850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:03.153850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:03.153850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:03.153850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:23:03.153850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:23:03.153850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:23:03.153850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:03.155073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:03.156205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:03.156205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:03.156205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:23:03.156205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:23:03.156205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:23:03.156205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:23:03.156205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:23:03.156205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:23:03.156205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:03.156205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:23:03.156205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 612/1239] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:23:03.161835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fd3c3-58a0-42ba-98de-0e92fd753397 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:03.281444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18fd3c3-58a0-42ba-98de-0e92fd753397 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:03.317785 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7ef4c-7a2e-43b1-a3e6-8bff83fbb136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:03.318396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7ef4c-7a2e-43b1-a3e6-8bff83fbb136 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:03.318680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7ef4c-7a2e-43b1-a3e6-8bff83fbb136 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:03.318962 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7ef4c-7a2e-43b1-a3e6-8bff83fbb136 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:03.319718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7ef4c-7a2e-43b1-a3e6-8bff83fbb136 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:03.320030 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7ef4c-7a2e-43b1-a3e6-8bff83fbb136 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:03.321461 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7ef4c-7a2e-43b1-a3e6-8bff83fbb136 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:03.341202 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 628/1240] 10.222.0.58 () {56 vars in 1143 bytes} [Wed Aug 24 12:23:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:03.347144 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570a7b8-a476-475e-bb93-a2ed0b73aa3f None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:03.465827 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570a7b8-a476-475e-bb93-a2ed0b73aa3f None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:03.505414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a351d-5e30-42f7-a115-273cc7b9840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:03.506024 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a351d-5e30-42f7-a115-273cc7b9840e None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:03.506309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a351d-5e30-42f7-a115-273cc7b9840e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:03.506584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a351d-5e30-42f7-a115-273cc7b9840e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:03.574714 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:03.582713 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a351d-5e30-42f7-a115-273cc7b9840e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:03.584657 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a351d-5e30-42f7-a115-273cc7b9840e None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:03.705964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 613/1241] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:23:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 12:23:03.712573 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7ef4c-7a2e-43b1-a3e6-8bff83fbb136 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:03.749127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:03.836587 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7ef4c-7a2e-43b1-a3e6-8bff83fbb136 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:03.879422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c60dd2-fcc7-4153-8d0e-bc95ffebbaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:03.879422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c60dd2-fcc7-4153-8d0e-bc95ffebbaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:03.879422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c60dd2-fcc7-4153-8d0e-bc95ffebbaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:03.879422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c60dd2-fcc7-4153-8d0e-bc95ffebbaa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:03.890008 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c60dd2-fcc7-4153-8d0e-bc95ffebbaa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:03.891466 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c60dd2-fcc7-4153-8d0e-bc95ffebbaa6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:03.906486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-b8c60dd2-fcc7-4153-8d0e-bc95ffebbaa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:03.906486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:03.906486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:03.906486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:03.906486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:03.906486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:03.906486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:03.906486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:03.906486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:03.906486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:03.906486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:03.906486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:03.906486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:03.906486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:03.906486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:03.906486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:03.907731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:03.909129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:03.909129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:03.909129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:23:03.909129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 629/1242] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:23:03.914343 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a351d-5e30-42f7-a115-273cc7b9840e None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:04.013774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:04.020031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a351d-5e30-42f7-a115-273cc7b9840e None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:04.056995 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aece7c-bc91-40c1-b2d8-e1006bf86228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:04.057750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aece7c-bc91-40c1-b2d8-e1006bf86228 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:04.058158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aece7c-bc91-40c1-b2d8-e1006bf86228 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:04.058598 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aece7c-bc91-40c1-b2d8-e1006bf86228 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:04.059374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aece7c-bc91-40c1-b2d8-e1006bf86228 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:04.059817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aece7c-bc91-40c1-b2d8-e1006bf86228 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:04.061366 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9aece7c-bc91-40c1-b2d8-e1006bf86228 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:04.080743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 614/1243] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:04.080958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c60dd2-fcc7-4153-8d0e-bc95ffebbaa6 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:04.164521 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:04.192709 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c60dd2-fcc7-4153-8d0e-bc95ffebbaa6 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:04.234343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fbbf0-f897-47e7-a5cb-d770f8c445e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:04.235107 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fbbf0-f897-47e7-a5cb-d770f8c445e4 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:04.235435 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fbbf0-f897-47e7-a5cb-d770f8c445e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:04.235761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fbbf0-f897-47e7-a5cb-d770f8c445e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:04.244176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62fbbf0-f897-47e7-a5cb-d770f8c445e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:04.245899 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62fbbf0-f897-47e7-a5cb-d770f8c445e4 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:04.259517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a62fbbf0-f897-47e7-a5cb-d770f8c445e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:04.259517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:04.259517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:04.259517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:04.259517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:04.259517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:04.259517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:04.259517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:04.259517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:04.259517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:04.259517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:04.259517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:04.259517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:23:04.259517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:23:04.259517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:04.260740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:04.262150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:23:04.262150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:23:04.262150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:23:04.262150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:23:04.262150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:23:04.262150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:23:04.262150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:04.262150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:23:04.262150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 630/1244] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:23:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:23:04.266842 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aece7c-bc91-40c1-b2d8-e1006bf86228 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:04.386583 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aece7c-bc91-40c1-b2d8-e1006bf86228 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:04.424384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985561a-440d-4411-9d2b-9b77db1d5221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:04.425001 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985561a-440d-4411-9d2b-9b77db1d5221 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:04.425883 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985561a-440d-4411-9d2b-9b77db1d5221 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:04.426217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985561a-440d-4411-9d2b-9b77db1d5221 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:04.427026 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985561a-440d-4411-9d2b-9b77db1d5221 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:04.427339 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985561a-440d-4411-9d2b-9b77db1d5221 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:04.428828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985561a-440d-4411-9d2b-9b77db1d5221 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:04.442479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 615/1245] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:23:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:04.448914 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fbbf0-f897-47e7-a5cb-d770f8c445e4 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:04.563728 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fbbf0-f897-47e7-a5cb-d770f8c445e4 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:04.603203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bd24e-a30e-41f7-aeb1-e8356055de66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:04.603904 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bd24e-a30e-41f7-aeb1-e8356055de66 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:04.604280 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bd24e-a30e-41f7-aeb1-e8356055de66 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:04.604566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bd24e-a30e-41f7-aeb1-e8356055de66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:04.605418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8bd24e-a30e-41f7-aeb1-e8356055de66 None admin] RBAC: Adding query filter params (role.id=5a12926bd458420e81e72ecf825d67ae, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=b7ba45e3cff74962965b248f9f017f11) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:04.605807 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8bd24e-a30e-41f7-aeb1-e8356055de66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:04.607613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8bd24e-a30e-41f7-aeb1-e8356055de66 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:04.614599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 631/1246] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:23:04 2022] GET /identity/v3/role_assignments?user.id=b7ba45e3cff74962965b248f9f017f11&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=5a12926bd458420e81e72ecf825d67ae => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:06.817656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39d22b-10cc-4e27-b65b-fa327775b3a9 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:06.818260 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39d22b-10cc-4e27-b65b-fa327775b3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:06.818260 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c39d22b-10cc-4e27-b65b-fa327775b3a9 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:06.820089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 616/1247] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:23:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:06.825446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009523ef-03a8-4c41-9b17-5c834dc83afb None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:06.825899 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009523ef-03a8-4c41-9b17-5c834dc83afb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:06.826000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009523ef-03a8-4c41-9b17-5c834dc83afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:06.833368 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:06.846396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-009523ef-03a8-4c41-9b17-5c834dc83afb None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:06.847669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-009523ef-03a8-4c41-9b17-5c834dc83afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:06.922364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 632/1248] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:23:07.240761 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39d22b-10cc-4e27-b65b-fa327775b3a9 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:07.264669 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:07.369804 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c39d22b-10cc-4e27-b65b-fa327775b3a9 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:07.408662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b32c9-78fa-49c9-9999-9b847f5b1cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:07.410379 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b32c9-78fa-49c9-9999-9b847f5b1cfc None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:07.410379 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b32c9-78fa-49c9-9999-9b847f5b1cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:07.410810 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b32c9-78fa-49c9-9999-9b847f5b1cfc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:07.422441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565b32c9-78fa-49c9-9999-9b847f5b1cfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:23:07.427474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-565b32c9-78fa-49c9-9999-9b847f5b1cfc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:23:07.427474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:07.427474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:07.427474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:07.427474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:07.427474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:07.427474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:07.427474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:07.427474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:07.427474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:07.427474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:07.427474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:07.427474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:23:07.427474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:23:07.427474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:23:07.427474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:23:07.428771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:23:07.428771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:23:07.428771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:23:07.428771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:23:07.428771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:07.428771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:07.428771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:07.428771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:07.428771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:07.428771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:07.428771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:07.428771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:07.428771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:07.428771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:07.428771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:07.428771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:07.429894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:07.429894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:07.429894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:07.429894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:07.429894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:23:07.429894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:23:07.429894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:23:07.429894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:23:07.429894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:23:07.429894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:23:07.429894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:23:07.429894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:23:07.429894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 617/1249] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:23:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:07.434747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009523ef-03a8-4c41-9b17-5c834dc83afb None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:07.549877 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009523ef-03a8-4c41-9b17-5c834dc83afb None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:07.584801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:07.590748 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fbadf1-4dbf-4c76-ba84-e6c9000cebf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:07.591391 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbadf1-4dbf-4c76-ba84-e6c9000cebf8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:07.591698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbadf1-4dbf-4c76-ba84-e6c9000cebf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:07.592016 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbadf1-4dbf-4c76-ba84-e6c9000cebf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:07.592747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fbadf1-4dbf-4c76-ba84-e6c9000cebf8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:07.593089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fbadf1-4dbf-4c76-ba84-e6c9000cebf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:07.594666 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fbadf1-4dbf-4c76-ba84-e6c9000cebf8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:07.603930 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 633/1250] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:23:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:07.609726 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b32c9-78fa-49c9-9999-9b847f5b1cfc None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:07.695760 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:07.722739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b32c9-78fa-49c9-9999-9b847f5b1cfc None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:07.761265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a0cd7-fb82-4b70-ae9c-4288b4e5797e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:07.761951 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a0cd7-fb82-4b70-ae9c-4288b4e5797e None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:07.762235 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a0cd7-fb82-4b70-ae9c-4288b4e5797e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:07.762516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85a0cd7-fb82-4b70-ae9c-4288b4e5797e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:07.774045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a0cd7-fb82-4b70-ae9c-4288b4e5797e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:07.775594 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85a0cd7-fb82-4b70-ae9c-4288b4e5797e None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:07.791143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-f85a0cd7-fb82-4b70-ae9c-4288b4e5797e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:07.791143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:07.791143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:07.791143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:07.791143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:07.791143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:07.791143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:07.791143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:07.791143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:07.791143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:07.791143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:07.791143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:07.791143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:07.791143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:07.791143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:07.791143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:07.792618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:07.793981 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:07.793981 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:07.793981 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:23:07.793981 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 618/1251] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:07.798819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fbadf1-4dbf-4c76-ba84-e6c9000cebf8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:07.919246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fbadf1-4dbf-4c76-ba84-e6c9000cebf8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:07.956363 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06edbd-edc0-498b-a9ae-d4fc90341565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:07.957056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06edbd-edc0-498b-a9ae-d4fc90341565 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:07.957254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06edbd-edc0-498b-a9ae-d4fc90341565 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:07.957528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d06edbd-edc0-498b-a9ae-d4fc90341565 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:07.958165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06edbd-edc0-498b-a9ae-d4fc90341565 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:07.958466 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06edbd-edc0-498b-a9ae-d4fc90341565 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:07.959882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d06edbd-edc0-498b-a9ae-d4fc90341565 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:07.971465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 634/1252] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:07.977176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a0cd7-fb82-4b70-ae9c-4288b4e5797e None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:08.089387 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85a0cd7-fb82-4b70-ae9c-4288b4e5797e None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:08.119090 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:08.128053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d4c11-0905-49ed-a705-124ba5bec763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:08.129602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d4c11-0905-49ed-a705-124ba5bec763 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:08.129849 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d4c11-0905-49ed-a705-124ba5bec763 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:08.130200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d4c11-0905-49ed-a705-124ba5bec763 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:08.138121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d4c11-0905-49ed-a705-124ba5bec763 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:08.139522 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d4c11-0905-49ed-a705-124ba5bec763 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:08.154487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-de6d4c11-0905-49ed-a705-124ba5bec763 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:08.154487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:08.154487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:08.154487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:08.154487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:08.154487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:08.154487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:08.154487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:08.154487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:08.154487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:08.154487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:08.154487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:08.154487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:23:08.154487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:23:08.154487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:23:08.154487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:08.155756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:08.156989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:08.156989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:08.156989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:23:08.156989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:23:08.156989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:23:08.156989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:23:08.156989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:23:08.156989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:23:08.156989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:08.156989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:23:08.156989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 619/1253] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:08.164480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06edbd-edc0-498b-a9ae-d4fc90341565 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:08.233921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:08.293045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d06edbd-edc0-498b-a9ae-d4fc90341565 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:08.330308 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47e22e-7376-445c-8586-828ff8cd9d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:08.330912 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47e22e-7376-445c-8586-828ff8cd9d94 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:08.331226 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47e22e-7376-445c-8586-828ff8cd9d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:08.331496 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee47e22e-7376-445c-8586-828ff8cd9d94 None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:08.332371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47e22e-7376-445c-8586-828ff8cd9d94 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:08.332683 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47e22e-7376-445c-8586-828ff8cd9d94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:08.334190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee47e22e-7376-445c-8586-828ff8cd9d94 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:08.352432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 635/1254] 10.222.0.58 () {56 vars in 1143 bytes} [Wed Aug 24 12:23:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:08.358847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d4c11-0905-49ed-a705-124ba5bec763 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:08.470971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d4c11-0905-49ed-a705-124ba5bec763 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:08.507341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f92a9-ff3d-422e-a972-08eb2275d178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:08.507985 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f92a9-ff3d-422e-a972-08eb2275d178 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:08.508269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f92a9-ff3d-422e-a972-08eb2275d178 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:08.508544 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f92a9-ff3d-422e-a972-08eb2275d178 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:08.568342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:08.583505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f92a9-ff3d-422e-a972-08eb2275d178 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:08.584976 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f92a9-ff3d-422e-a972-08eb2275d178 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:08.695960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 620/1255] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:23:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:23:08.702123 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47e22e-7376-445c-8586-828ff8cd9d94 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:08.816226 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee47e22e-7376-445c-8586-828ff8cd9d94 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:08.837453 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:08.859933 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26932e32-5a44-4767-aa8b-26bd22befb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:08.860570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26932e32-5a44-4767-aa8b-26bd22befb65 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:08.860851 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26932e32-5a44-4767-aa8b-26bd22befb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:08.861127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26932e32-5a44-4767-aa8b-26bd22befb65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:08.873566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26932e32-5a44-4767-aa8b-26bd22befb65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:08.875309 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26932e32-5a44-4767-aa8b-26bd22befb65 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:08.891532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-26932e32-5a44-4767-aa8b-26bd22befb65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:08.891532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:08.891532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:08.891532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:08.891532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:08.891532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:08.891532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:08.891532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:08.891532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:08.891532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:08.891532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:08.891532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:08.891532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:08.891532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:08.891532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:08.891532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:08.892682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:08.893589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:08.893589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:08.893589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:23:08.893589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 636/1256] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:08.898580 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f92a9-ff3d-422e-a972-08eb2275d178 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:09.002682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:09.017380 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f92a9-ff3d-422e-a972-08eb2275d178 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:09.055117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb802eb-e2f4-4fa0-aa0c-1a7bd84128e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:09.055922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb802eb-e2f4-4fa0-aa0c-1a7bd84128e4 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:09.056187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb802eb-e2f4-4fa0-aa0c-1a7bd84128e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:09.056449 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb802eb-e2f4-4fa0-aa0c-1a7bd84128e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:09.057166 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb802eb-e2f4-4fa0-aa0c-1a7bd84128e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:09.057491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb802eb-e2f4-4fa0-aa0c-1a7bd84128e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:09.059044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb802eb-e2f4-4fa0-aa0c-1a7bd84128e4 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:09.071350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 621/1257] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:09.077576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26932e32-5a44-4767-aa8b-26bd22befb65 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:09.192098 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26932e32-5a44-4767-aa8b-26bd22befb65 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:09.233118 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d6772-a865-424a-978c-518725a2a9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:09.233773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d6772-a865-424a-978c-518725a2a9cd None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:09.234039 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d6772-a865-424a-978c-518725a2a9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:09.234312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d6772-a865-424a-978c-518725a2a9cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:09.242449 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d6772-a865-424a-978c-518725a2a9cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:09.244056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547d6772-a865-424a-978c-518725a2a9cd None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:09.256215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-547d6772-a865-424a-978c-518725a2a9cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:09.256215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:09.256215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:09.256215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:09.256215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:09.256215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:09.256215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:09.256215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:09.256215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:09.256215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:09.256215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:09.256215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:09.256215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:23:09.256215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:23:09.256215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:09.257436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:09.258651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:23:09.258651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:23:09.258651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:23:09.258651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:23:09.258651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:23:09.258651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:23:09.258651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:09.258651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:23:09.258651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 637/1258] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:23:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:09.263382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb802eb-e2f4-4fa0-aa0c-1a7bd84128e4 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:09.375510 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb802eb-e2f4-4fa0-aa0c-1a7bd84128e4 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:09.413708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf51d5-2845-44f0-9661-23134bc8fe33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:09.414335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf51d5-2845-44f0-9661-23134bc8fe33 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:09.414675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf51d5-2845-44f0-9661-23134bc8fe33 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:09.414953 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf51d5-2845-44f0-9661-23134bc8fe33 None admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:09.415704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf51d5-2845-44f0-9661-23134bc8fe33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:09.415973 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf51d5-2845-44f0-9661-23134bc8fe33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:09.417413 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf51d5-2845-44f0-9661-23134bc8fe33 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:09.430807 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 622/1259] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:23:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:23:09.437178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d6772-a865-424a-978c-518725a2a9cd None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:09.466896 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:09.552143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d6772-a865-424a-978c-518725a2a9cd None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:09.587951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03de53a-91bf-4f98-a211-c80ffa022fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:09.588572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03de53a-91bf-4f98-a211-c80ffa022fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:09.588848 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03de53a-91bf-4f98-a211-c80ffa022fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:09.589115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03de53a-91bf-4f98-a211-c80ffa022fa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:09.589849 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03de53a-91bf-4f98-a211-c80ffa022fa3 None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=b7ba45e3cff74962965b248f9f017f11) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:09.590154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03de53a-91bf-4f98-a211-c80ffa022fa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:09.591654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03de53a-91bf-4f98-a211-c80ffa022fa3 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:09.598376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 638/1260] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:23:09 2022] GET /identity/v3/role_assignments?user.id=b7ba45e3cff74962965b248f9f017f11&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:11.556827 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed0a18-0f83-4d37-a76e-b4af8c76d667 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:11.557381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed0a18-0f83-4d37-a76e-b4af8c76d667 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:11.557381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed0a18-0f83-4d37-a76e-b4af8c76d667 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:11.558721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 623/1261] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:11.564570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9b07b-dce1-4b68-a9ac-8c4c890cbbef None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:11.564969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9b07b-dce1-4b68-a9ac-8c4c890cbbef None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:11.565121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9b07b-dce1-4b68-a9ac-8c4c890cbbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:11.583465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6ba9b07b-dce1-4b68-a9ac-8c4c890cbbef None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:11.584683 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6ba9b07b-dce1-4b68-a9ac-8c4c890cbbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:11.649733 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 639/1262] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:23:11.917106 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed0a18-0f83-4d37-a76e-b4af8c76d667 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:11.928104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:12.017127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed0a18-0f83-4d37-a76e-b4af8c76d667 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:12.055928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f1ced-e00f-45c8-83cd-65234e1302ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:12.056549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f1ced-e00f-45c8-83cd-65234e1302ec None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:12.056918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f1ced-e00f-45c8-83cd-65234e1302ec None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:12.057198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f1ced-e00f-45c8-83cd-65234e1302ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:12.067862 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f1ced-e00f-45c8-83cd-65234e1302ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:23:12.072556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-935f1ced-e00f-45c8-83cd-65234e1302ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:23:12.072556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:12.072556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:12.072556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:12.072556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:12.072556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:12.072556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:12.072556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:12.072556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:12.072556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:12.072556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:12.072556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:12.072556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:23:12.072556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:23:12.072556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:23:12.072556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:23:12.073918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:23:12.073918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:23:12.073918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:23:12.073918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:23:12.073918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:12.073918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:12.073918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:12.073918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:12.073918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:12.073918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:12.073918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:12.073918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:12.073918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:12.073918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:12.073918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:12.073918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:12.074972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:12.074972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:12.074972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:12.074972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:12.074972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:23:12.074972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:23:12.074972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:23:12.074972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:23:12.074972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:23:12.074972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:23:12.074972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:23:12.074972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:23:12.074972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 624/1263] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:23:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:12.079923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9b07b-dce1-4b68-a9ac-8c4c890cbbef None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:12.098376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:12.177348 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9b07b-dce1-4b68-a9ac-8c4c890cbbef None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:12.215075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a41bae-3b4b-49cd-8347-f616e4291ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:12.215791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a41bae-3b4b-49cd-8347-f616e4291ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:12.216102 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a41bae-3b4b-49cd-8347-f616e4291ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:12.216346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a41bae-3b4b-49cd-8347-f616e4291ec8 None admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:12.217008 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a41bae-3b4b-49cd-8347-f616e4291ec8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:12.217311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a41bae-3b4b-49cd-8347-f616e4291ec8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:12.218736 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a41bae-3b4b-49cd-8347-f616e4291ec8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:12.227862 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 640/1264] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:23:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:12.233967 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f1ced-e00f-45c8-83cd-65234e1302ec None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:12.308273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:12.341214 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f1ced-e00f-45c8-83cd-65234e1302ec None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:12.373291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5bfa2-ba48-4b5a-9854-a7ccc0be2ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:12.373913 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5bfa2-ba48-4b5a-9854-a7ccc0be2ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:12.374195 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5bfa2-ba48-4b5a-9854-a7ccc0be2ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:12.374467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5bfa2-ba48-4b5a-9854-a7ccc0be2ec7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:12.384742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5bfa2-ba48-4b5a-9854-a7ccc0be2ec7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:12.386181 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5bfa2-ba48-4b5a-9854-a7ccc0be2ec7 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:12.402080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-eaa5bfa2-ba48-4b5a-9854-a7ccc0be2ec7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:12.402080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:12.402080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:12.402080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:12.402080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:12.402080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:12.402080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:12.402080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:12.402080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:12.402080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:12.402080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:12.402080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:12.402080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:12.402080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:12.402080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:12.402080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:12.403986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:12.405237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:12.405237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:12.405237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:23:12.405237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 625/1265] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:23:12.410409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a41bae-3b4b-49cd-8347-f616e4291ec8 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:12.493721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:12.512880 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a41bae-3b4b-49cd-8347-f616e4291ec8 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:12.545251 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b32d4-ac8c-4290-a53e-9763694e9645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:12.545849 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b32d4-ac8c-4290-a53e-9763694e9645 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:12.546133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b32d4-ac8c-4290-a53e-9763694e9645 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:12.546408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b32d4-ac8c-4290-a53e-9763694e9645 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:12.547043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b32d4-ac8c-4290-a53e-9763694e9645 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:12.547345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b32d4-ac8c-4290-a53e-9763694e9645 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:12.548752 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b32d4-ac8c-4290-a53e-9763694e9645 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:12.560687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 641/1266] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:12.567096 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5bfa2-ba48-4b5a-9854-a7ccc0be2ec7 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:12.662847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5bfa2-ba48-4b5a-9854-a7ccc0be2ec7 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:12.681926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:12.696906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2a518-a645-4df9-ad09-9e5d95b46b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:12.697530 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a518-a645-4df9-ad09-9e5d95b46b27 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:12.697814 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a518-a645-4df9-ad09-9e5d95b46b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:12.698087 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a518-a645-4df9-ad09-9e5d95b46b27 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:12.705968 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2a518-a645-4df9-ad09-9e5d95b46b27 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:12.707564 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2a518-a645-4df9-ad09-9e5d95b46b27 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:12.718934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-30a2a518-a645-4df9-ad09-9e5d95b46b27 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:12.718934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:12.718934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:12.718934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:12.718934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:12.718934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:12.718934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:12.718934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:12.718934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:12.718934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:12.718934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:12.718934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:12.718934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:23:12.718934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:23:12.718934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:23:12.718934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:12.724868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:12.726073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:12.726073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:12.726073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:23:12.726073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:23:12.726073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:23:12.726073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:23:12.726073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:23:12.726073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:23:12.726073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:12.726073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:23:12.726073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 626/1267] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:12.726805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b32d4-ac8c-4290-a53e-9763694e9645 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:12.825347 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b32d4-ac8c-4290-a53e-9763694e9645 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:12.858120 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a4ecf-3ba8-4586-a905-110c8e2b43be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:12.858712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a4ecf-3ba8-4586-a905-110c8e2b43be None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:12.858994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a4ecf-3ba8-4586-a905-110c8e2b43be None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:12.859268 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a4ecf-3ba8-4586-a905-110c8e2b43be None admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:12.860012 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a4ecf-3ba8-4586-a905-110c8e2b43be None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:12.860318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a4ecf-3ba8-4586-a905-110c8e2b43be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:12.861745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a4ecf-3ba8-4586-a905-110c8e2b43be None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:12.870760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:12.879896 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 642/1268] 10.222.0.58 () {56 vars in 1143 bytes} [Wed Aug 24 12:23:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:12.885317 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2a518-a645-4df9-ad09-9e5d95b46b27 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:12.982495 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2a518-a645-4df9-ad09-9e5d95b46b27 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:13.014833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ea75d-ced1-4de1-bbc9-518308a1f51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:13.015454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ea75d-ced1-4de1-bbc9-518308a1f51f None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:13.015764 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ea75d-ced1-4de1-bbc9-518308a1f51f None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:13.016035 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ea75d-ced1-4de1-bbc9-518308a1f51f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:13.027575 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ea75d-ced1-4de1-bbc9-518308a1f51f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:13.029041 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ea75d-ced1-4de1-bbc9-518308a1f51f None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:13.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-f16ea75d-ced1-4de1-bbc9-518308a1f51f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:13.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:13.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:13.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:13.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:13.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:13.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:13.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:13.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:13.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:13.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:13.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:13.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:13.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:13.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:13.045938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:13.047107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:13.048210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:13.048210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:13.048210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:23:13.048210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 627/1269] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:13.054425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a4ecf-3ba8-4586-a905-110c8e2b43be None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:13.157566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a4ecf-3ba8-4586-a905-110c8e2b43be None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:13.193172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed86af-0362-4fe5-8714-dde3eee6beb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:13.193826 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed86af-0362-4fe5-8714-dde3eee6beb5 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:13.194156 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed86af-0362-4fe5-8714-dde3eee6beb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:13.194478 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed86af-0362-4fe5-8714-dde3eee6beb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:13.195168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed86af-0362-4fe5-8714-dde3eee6beb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:13.195501 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed86af-0362-4fe5-8714-dde3eee6beb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:13.197069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed86af-0362-4fe5-8714-dde3eee6beb5 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:13.213124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 643/1270] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:13.219254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ea75d-ced1-4de1-bbc9-518308a1f51f None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:13.256642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:13.334582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ea75d-ced1-4de1-bbc9-518308a1f51f None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:13.367956 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31539e8e-b09c-4ef2-85b2-0e29472a5332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:13.368584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31539e8e-b09c-4ef2-85b2-0e29472a5332 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:13.368860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31539e8e-b09c-4ef2-85b2-0e29472a5332 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:13.369129 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31539e8e-b09c-4ef2-85b2-0e29472a5332 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:13.375486 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31539e8e-b09c-4ef2-85b2-0e29472a5332 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:13.376963 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31539e8e-b09c-4ef2-85b2-0e29472a5332 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:13.387746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-31539e8e-b09c-4ef2-85b2-0e29472a5332 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:13.387746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:13.387746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:13.387746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:13.387746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:13.387746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:13.387746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:13.387746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:13.387746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:13.387746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:13.387746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:13.387746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:13.387746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:23:13.387746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:23:13.387746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:13.388834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:13.390055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:23:13.390055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:23:13.390055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:23:13.390055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:23:13.390055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:23:13.390055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:23:13.390055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:13.390055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:23:13.391781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 628/1271] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:23:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:13.395204 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed86af-0362-4fe5-8714-dde3eee6beb5 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:13.457655 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:13.501997 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed86af-0362-4fe5-8714-dde3eee6beb5 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:13.535387 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad35b3ca-efc6-4e6b-bd30-4a49464bbef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:13.536015 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35b3ca-efc6-4e6b-bd30-4a49464bbef0 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:13.536295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35b3ca-efc6-4e6b-bd30-4a49464bbef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:13.536565 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad35b3ca-efc6-4e6b-bd30-4a49464bbef0 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:13.537248 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35b3ca-efc6-4e6b-bd30-4a49464bbef0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:13.537551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35b3ca-efc6-4e6b-bd30-4a49464bbef0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:13.538974 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad35b3ca-efc6-4e6b-bd30-4a49464bbef0 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:13.550524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 644/1272] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:23:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:13.556702 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31539e8e-b09c-4ef2-85b2-0e29472a5332 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:13.646536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:13.655785 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31539e8e-b09c-4ef2-85b2-0e29472a5332 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:13.688442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce872a-7778-443f-b7e6-74dee70a0e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:13.689092 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce872a-7778-443f-b7e6-74dee70a0e7c None admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:13.689374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce872a-7778-443f-b7e6-74dee70a0e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:13.689650 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce872a-7778-443f-b7e6-74dee70a0e7c None admin] PATH_INFO: `/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/b7ba45e3cff74962965b248f9f017f11/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:13.692084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce872a-7778-443f-b7e6-74dee70a0e7c None admin] RBAC: Authorizing `identity:create_grant(project_id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user_id=b7ba45e3cff74962965b248f9f017f11, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:13.693517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce872a-7778-443f-b7e6-74dee70a0e7c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:13.707809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 629/1273] 10.222.0.58 () {58 vars in 1439 bytes} [Wed Aug 24 12:23:13 2022] PUT /identity/v3/projects/7fb9c0b20eea4e6dab2cb9fe0b9d677b/users/b7ba45e3cff74962965b248f9f017f11/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:23:15.576743 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c38d7a-a839-47a4-9656-7dd769d4d3fb None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:15.577280 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c38d7a-a839-47a4-9656-7dd769d4d3fb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:15.577280 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c38d7a-a839-47a4-9656-7dd769d4d3fb None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:15.578651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 645/1274] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:15.584401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d68f4-6753-4493-8db2-c30c1c14c2f1 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:15.584840 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d68f4-6753-4493-8db2-c30c1c14c2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:15.584942 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d68f4-6753-4493-8db2-c30c1c14c2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:15.604647 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-ca6d68f4-6753-4493-8db2-c30c1c14c2f1 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:15.605840 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-ca6d68f4-6753-4493-8db2-c30c1c14c2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:15.670506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 630/1275] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:23:15.942286 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c38d7a-a839-47a4-9656-7dd769d4d3fb None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:16.041947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c38d7a-a839-47a4-9656-7dd769d4d3fb None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:16.058441 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:16.076277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d1b43-aebc-4690-b044-9fd65cceb49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:16.076893 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d1b43-aebc-4690-b044-9fd65cceb49a None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:16.077177 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d1b43-aebc-4690-b044-9fd65cceb49a None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:16.077452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d1b43-aebc-4690-b044-9fd65cceb49a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:16.087820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d1b43-aebc-4690-b044-9fd65cceb49a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 12:23:16.092536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-e23d1b43-aebc-4690-b044-9fd65cceb49a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:23:16.092536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:16.092536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:16.092536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:16.092536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:16.092536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:16.092536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:16.092536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:16.092536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:16.092536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:16.092536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:16.092536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:16.092536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 12:23:16.092536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 12:23:16.092536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 12:23:16.092536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 12:23:16.093737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 12:23:16.093737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 12:23:16.093737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 12:23:16.093737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 12:23:16.093737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:16.093737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:16.093737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:16.093737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:16.093737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:16.093737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:16.093737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:16.093737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:16.093737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:16.093737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:16.093737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:16.093737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:16.094605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:16.094605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:16.094605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:16.094605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:16.094605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 12:23:16.094605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 12:23:16.094605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 12:23:16.094605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 12:23:16.094605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 12:23:16.094605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 12:23:16.094605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 12:23:16.094605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:23:16.094605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 646/1276] 10.222.0.58 () {56 vars in 1096 bytes} [Wed Aug 24 12:23:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 12:23:16.099531 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d68f4-6753-4493-8db2-c30c1c14c2f1 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:16.177882 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:16.199042 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d68f4-6753-4493-8db2-c30c1c14c2f1 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:16.242464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512e83c-5583-4a10-8741-ceeb14d6d21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:16.243205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512e83c-5583-4a10-8741-ceeb14d6d21e None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:16.243557 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512e83c-5583-4a10-8741-ceeb14d6d21e None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:16.243918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512e83c-5583-4a10-8741-ceeb14d6d21e None admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:16.244718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512e83c-5583-4a10-8741-ceeb14d6d21e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:16.245132 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512e83c-5583-4a10-8741-ceeb14d6d21e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:16.246961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512e83c-5583-4a10-8741-ceeb14d6d21e None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:16.257670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 631/1277] 10.222.0.58 () {56 vars in 1099 bytes} [Wed Aug 24 12:23:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:16.264044 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d1b43-aebc-4690-b044-9fd65cceb49a None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:16.363371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d1b43-aebc-4690-b044-9fd65cceb49a None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:16.397051 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d220a-7c0b-433c-877b-29bb2d96a11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:16.397673 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d220a-7c0b-433c-877b-29bb2d96a11e None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:16.397958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d220a-7c0b-433c-877b-29bb2d96a11e None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:16.398232 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7d220a-7c0b-433c-877b-29bb2d96a11e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:16.407988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d220a-7c0b-433c-877b-29bb2d96a11e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:16.409504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7d220a-7c0b-433c-877b-29bb2d96a11e None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:16.423797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-3c7d220a-7c0b-433c-877b-29bb2d96a11e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:16.423797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:16.423797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:16.423797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:16.423797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:16.423797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:16.423797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:16.423797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:16.423797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:16.423797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:16.423797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:16.423797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:16.423797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:16.423797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:16.423797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:16.423797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:16.425463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:16.426463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:16.426463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:16.426463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:23:16.426463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 647/1278] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:16.434376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512e83c-5583-4a10-8741-ceeb14d6d21e None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:16.532407 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512e83c-5583-4a10-8741-ceeb14d6d21e None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:16.565493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982aa8c9-c4b2-4aca-8acf-af33286ae270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:16.566089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982aa8c9-c4b2-4aca-8acf-af33286ae270 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:16.566368 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982aa8c9-c4b2-4aca-8acf-af33286ae270 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:16.566643 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982aa8c9-c4b2-4aca-8acf-af33286ae270 None admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:16.567282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982aa8c9-c4b2-4aca-8acf-af33286ae270 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:16.567586 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982aa8c9-c4b2-4aca-8acf-af33286ae270 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:16.568982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982aa8c9-c4b2-4aca-8acf-af33286ae270 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:16.580561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 632/1279] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:16.586196 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d220a-7c0b-433c-877b-29bb2d96a11e None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:16.617212 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:16.686415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7d220a-7c0b-433c-877b-29bb2d96a11e None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:16.719394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e3172-3998-471a-bd8c-36b8c7d9f0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:16.720050 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e3172-3998-471a-bd8c-36b8c7d9f0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:16.720319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e3172-3998-471a-bd8c-36b8c7d9f0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:16.720592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e3172-3998-471a-bd8c-36b8c7d9f0c9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:16.727979 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e3172-3998-471a-bd8c-36b8c7d9f0c9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:16.729411 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e3172-3998-471a-bd8c-36b8c7d9f0c9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:16.740335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-be9e3172-3998-471a-bd8c-36b8c7d9f0c9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:16.740335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:16.740335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:16.740335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:16.740335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:16.740335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:16.740335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:16.740335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:16.740335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:16.740335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:16.740335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:16.740335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:16.740335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 12:23:16.740335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 12:23:16.740335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 12:23:16.740335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:16.741503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:16.742926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:16.742926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:16.742926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 12:23:16.742926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 12:23:16.742926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 12:23:16.742926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 12:23:16.742926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 12:23:16.742926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 12:23:16.742926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 12:23:16.742926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:23:16.742926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 648/1280] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:16.747234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982aa8c9-c4b2-4aca-8acf-af33286ae270 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:16.751035 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:16.845819 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982aa8c9-c4b2-4aca-8acf-af33286ae270 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:16.879591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f237f-0a4f-4c74-9104-8be2b5d744cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:16.880216 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f237f-0a4f-4c74-9104-8be2b5d744cd None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:16.880494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f237f-0a4f-4c74-9104-8be2b5d744cd None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:16.880761 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f237f-0a4f-4c74-9104-8be2b5d744cd None admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:16.881484 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f237f-0a4f-4c74-9104-8be2b5d744cd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:16.881788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f237f-0a4f-4c74-9104-8be2b5d744cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:16.883206 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f237f-0a4f-4c74-9104-8be2b5d744cd None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:16.900588 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 633/1281] 10.222.0.58 () {56 vars in 1143 bytes} [Wed Aug 24 12:23:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:16.906253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e3172-3998-471a-bd8c-36b8c7d9f0c9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:16.995262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:17.005335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e3172-3998-471a-bd8c-36b8c7d9f0c9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:17.038469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e7e5a-ace2-4115-9297-9f2791dec068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:17.039053 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e7e5a-ace2-4115-9297-9f2791dec068 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:17.039330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e7e5a-ace2-4115-9297-9f2791dec068 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:17.039601 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e7e5a-ace2-4115-9297-9f2791dec068 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:17.113311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e7e5a-ace2-4115-9297-9f2791dec068 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:17.114746 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e7e5a-ace2-4115-9297-9f2791dec068 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:17.201913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:17.214762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 649/1282] 10.222.0.58 () {58 vars in 1282 bytes} [Wed Aug 24 12:23:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 12:23:17.220606 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f237f-0a4f-4c74-9104-8be2b5d744cd None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:17.279659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:17.317160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f237f-0a4f-4c74-9104-8be2b5d744cd None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:17.349899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4db2d-fc4b-4e77-a4c5-241bc4c7f262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:17.350508 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4db2d-fc4b-4e77-a4c5-241bc4c7f262 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:17.350783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4db2d-fc4b-4e77-a4c5-241bc4c7f262 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:17.351052 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4db2d-fc4b-4e77-a4c5-241bc4c7f262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:17.360676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4db2d-fc4b-4e77-a4c5-241bc4c7f262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:17.362105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4db2d-fc4b-4e77-a4c5-241bc4c7f262 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:17.376894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-09f4db2d-fc4b-4e77-a4c5-241bc4c7f262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:17.376894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:17.376894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:17.376894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:17.376894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:17.376894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:17.376894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:17.376894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:17.376894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:17.376894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:17.376894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:17.376894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:17.376894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 12:23:17.376894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 12:23:17.376894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 12:23:17.376894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:17.378536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 12:23:17.379582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 12:23:17.379582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 12:23:17.379582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:23:17.379582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 634/1283] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:17.384183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e7e5a-ace2-4115-9297-9f2791dec068 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:17.481158 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e7e5a-ace2-4115-9297-9f2791dec068 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:17.514316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34801811-a3e7-48fd-b451-f3f1a72ac81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:17.514913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34801811-a3e7-48fd-b451-f3f1a72ac81f None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:17.515193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34801811-a3e7-48fd-b451-f3f1a72ac81f None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:17.515464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34801811-a3e7-48fd-b451-f3f1a72ac81f None admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:17.516127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34801811-a3e7-48fd-b451-f3f1a72ac81f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:17.516433 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34801811-a3e7-48fd-b451-f3f1a72ac81f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:17.517797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34801811-a3e7-48fd-b451-f3f1a72ac81f None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:17.528592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 650/1284] 10.222.0.58 () {56 vars in 1109 bytes} [Wed Aug 24 12:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:17.534725 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4db2d-fc4b-4e77-a4c5-241bc4c7f262 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:17.630790 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4db2d-fc4b-4e77-a4c5-241bc4c7f262 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:17.644408 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:17.664554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ade7e-8404-4f05-a0d2-aef57acffaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:17.665168 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ade7e-8404-4f05-a0d2-aef57acffaa8 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:17.665445 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ade7e-8404-4f05-a0d2-aef57acffaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:17.665716 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ade7e-8404-4f05-a0d2-aef57acffaa8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:17.672096 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ade7e-8404-4f05-a0d2-aef57acffaa8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:17.673528 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ade7e-8404-4f05-a0d2-aef57acffaa8 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:17.684376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application [None req-186ade7e-8404-4f05-a0d2-aef57acffaa8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:17.684376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:17.684376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:17.684376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:17.684376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:17.684376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:17.684376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:17.684376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:17.684376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:17.684376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:17.684376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:17.684376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:17.684376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 12:23:17.684376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 12:23:17.684376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:17.685466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:17.686613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 12:23:17.686613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 12:23:17.686613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 12:23:17.686613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 12:23:17.686613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 12:23:17.686613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 12:23:17.686613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 12:23:17.686613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: ERROR keystone.server.flask.application Aug 24 12:23:17.686613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 635/1285] 10.222.0.58 () {56 vars in 1111 bytes} [Wed Aug 24 12:23:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:17.691396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34801811-a3e7-48fd-b451-f3f1a72ac81f None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:17.747612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:17.796829 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34801811-a3e7-48fd-b451-f3f1a72ac81f None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:17.829826 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ec16a-e2eb-4c01-a813-b79b1655ffa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:17.830432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ec16a-e2eb-4c01-a813-b79b1655ffa6 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:17.830713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ec16a-e2eb-4c01-a813-b79b1655ffa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:17.831505 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ec16a-e2eb-4c01-a813-b79b1655ffa6 None admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:17.832237 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ec16a-e2eb-4c01-a813-b79b1655ffa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:17.832540 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ec16a-e2eb-4c01-a813-b79b1655ffa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:17.833972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ec16a-e2eb-4c01-a813-b79b1655ffa6 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:17.845507 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 651/1286] 10.222.0.58 () {56 vars in 1148 bytes} [Wed Aug 24 12:23:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:17.852264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ade7e-8404-4f05-a0d2-aef57acffaa8 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:17.953230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ade7e-8404-4f05-a0d2-aef57acffaa8 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:17.989395 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f0b3a-7f1d-4008-b735-6c43b3b0d6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:17.989994 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f0b3a-7f1d-4008-b735-6c43b3b0d6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:17.990276 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f0b3a-7f1d-4008-b735-6c43b3b0d6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:17.990594 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f0b3a-7f1d-4008-b735-6c43b3b0d6c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:17.991322 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f0b3a-7f1d-4008-b735-6c43b3b0d6c9 None admin] RBAC: Adding query filter params (role.id=6f69d065257c40daa87e7e1e460a17d9, scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b, user.id=b7ba45e3cff74962965b248f9f017f11) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:17.991655 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f0b3a-7f1d-4008-b735-6c43b3b0d6c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:17.993159 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f0b3a-7f1d-4008-b735-6c43b3b0d6c9 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:17.998959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 636/1287] 10.222.0.58 () {56 vars in 1374 bytes} [Wed Aug 24 12:23:17 2022] GET /identity/v3/role_assignments?user.id=b7ba45e3cff74962965b248f9f017f11&scope.project.id=7fb9c0b20eea4e6dab2cb9fe0b9d677b&role.id=6f69d065257c40daa87e7e1e460a17d9 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:19.916487 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f8560-b5c0-4dbe-844c-d2cc1cf6fba4 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:19.916964 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f8560-b5c0-4dbe-844c-d2cc1cf6fba4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:19.917055 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f8560-b5c0-4dbe-844c-d2cc1cf6fba4 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:19.918399 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 652/1288] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:19.923554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f278e-4015-4342-9b1a-d1b7a8d1437b None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:19.923990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f278e-4015-4342-9b1a-d1b7a8d1437b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:19.924136 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f278e-4015-4342-9b1a-d1b7a8d1437b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:19.942443 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-860f278e-4015-4342-9b1a-d1b7a8d1437b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:19.943653 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-860f278e-4015-4342-9b1a-d1b7a8d1437b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:19.974045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:20.010414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 637/1289] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:23:20.276935 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f8560-b5c0-4dbe-844c-d2cc1cf6fba4 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:20.377617 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f8560-b5c0-4dbe-844c-d2cc1cf6fba4 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:20.388593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:20.412264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ad0cc-66b0-47b3-af2c-b376cbc173e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:20.412883 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ad0cc-66b0-47b3-af2c-b376cbc173e9 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:20.413160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ad0cc-66b0-47b3-af2c-b376cbc173e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:20.413430 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ad0cc-66b0-47b3-af2c-b376cbc173e9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:20.413998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ad0cc-66b0-47b3-af2c-b376cbc173e9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:20.415350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ad0cc-66b0-47b3-af2c-b376cbc173e9 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-a44ad0cc-66b0-47b3-af2c-b376cbc173e9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:20.423838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:20.425067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:20.425067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:20.425067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:20.425067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:20.425067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:20.425067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:20.425067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:23:20.425067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:23:20.425067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:23:20.425067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:23:20.425067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:20.425067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:20.425067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:20.425067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:20.425067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:20.425067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:20.425965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:20.425965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:20.425965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:20.425965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:20.425965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:20.425965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:20.425965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:20.425965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:20.425965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:20.425965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:20.425965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:23:20.425965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:23:20.425965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:20.425965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:23:20.425965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 653/1290] 10.222.0.58 () {56 vars in 1117 bytes} [Wed Aug 24 12:23:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:20.431128 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f278e-4015-4342-9b1a-d1b7a8d1437b None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:20.527340 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f278e-4015-4342-9b1a-d1b7a8d1437b None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:20.559031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7fbbd5-44b1-42c6-87b4-c8e2be63b351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:20.559645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fbbd5-44b1-42c6-87b4-c8e2be63b351 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:20.559924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fbbd5-44b1-42c6-87b4-c8e2be63b351 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:20.560193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7fbbd5-44b1-42c6-87b4-c8e2be63b351 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:20.560810 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7fbbd5-44b1-42c6-87b4-c8e2be63b351 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:20.561119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7fbbd5-44b1-42c6-87b4-c8e2be63b351 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:20.562468 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7fbbd5-44b1-42c6-87b4-c8e2be63b351 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:20.567587 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 638/1291] 10.222.0.58 () {56 vars in 1116 bytes} [Wed Aug 24 12:23:20 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:20.573171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ad0cc-66b0-47b3-af2c-b376cbc173e9 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:20.676368 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ad0cc-66b0-47b3-af2c-b376cbc173e9 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:20.708914 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83ee51e-1272-4271-8e8f-f9a2987871d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:20.709505 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ee51e-1272-4271-8e8f-f9a2987871d8 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:20.709785 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ee51e-1272-4271-8e8f-f9a2987871d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:20.710057 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83ee51e-1272-4271-8e8f-f9a2987871d8 None admin] PATH_INFO: `/v3/services` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:20.710674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ee51e-1272-4271-8e8f-f9a2987871d8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:20.710977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ee51e-1272-4271-8e8f-f9a2987871d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:20.712361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83ee51e-1272-4271-8e8f-f9a2987871d8 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:20.716663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 654/1292] 10.222.0.58 () {56 vars in 1116 bytes} [Wed Aug 24 12:23:20 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:22.620572 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af52b0-af88-41af-a2b6-5011994cd055 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:22.621105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af52b0-af88-41af-a2b6-5011994cd055 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:22.621105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af52b0-af88-41af-a2b6-5011994cd055 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:22.622447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 639/1293] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:22.628080 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0f765-4aca-42cc-94c9-a3df87299de9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:22.628531 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0f765-4aca-42cc-94c9-a3df87299de9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:22.628654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da0f765-4aca-42cc-94c9-a3df87299de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:22.647073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6da0f765-4aca-42cc-94c9-a3df87299de9 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:22.648290 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6da0f765-4aca-42cc-94c9-a3df87299de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:22.680783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:22.717560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 655/1294] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:23:22.990260 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af52b0-af88-41af-a2b6-5011994cd055 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:23.020661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:23.102544 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af52b0-af88-41af-a2b6-5011994cd055 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:23.143153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31613f7a-85b6-4729-ab83-0d13642a0498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:23.144043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31613f7a-85b6-4729-ab83-0d13642a0498 None admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:23.144334 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31613f7a-85b6-4729-ab83-0d13642a0498 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:23.144611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31613f7a-85b6-4729-ab83-0d13642a0498 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:23.145254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31613f7a-85b6-4729-ab83-0d13642a0498 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:23.146624 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31613f7a-85b6-4729-ab83-0d13642a0498 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:23.160875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 640/1295] 10.222.0.58 () {60 vars in 1141 bytes} [Wed Aug 24 12:23:22 2022] POST /identity/v3/services => generated 246 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:23:25.097768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfed51b-ba37-4574-a5b6-0aecba99c475 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:25.098291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfed51b-ba37-4574-a5b6-0aecba99c475 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:25.098291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfed51b-ba37-4574-a5b6-0aecba99c475 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:25.099691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 656/1296] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:23:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:25.104910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e8f2f-9b61-4cf4-8c8e-a7c1dec83c5d None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:25.105301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e8f2f-9b61-4cf4-8c8e-a7c1dec83c5d None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:25.105471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e8f2f-9b61-4cf4-8c8e-a7c1dec83c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:25.130511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-767e8f2f-9b61-4cf4-8c8e-a7c1dec83c5d None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:25.131805 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-767e8f2f-9b61-4cf4-8c8e-a7c1dec83c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:25.204649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 641/1297] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:25 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:23:25.474195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfed51b-ba37-4574-a5b6-0aecba99c475 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:25.572157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfed51b-ba37-4574-a5b6-0aecba99c475 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:25.605153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d3461-a44b-4895-8d52-9a9bfc0ffb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:25.605768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d3461-a44b-4895-8d52-9a9bfc0ffb90 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:25.606049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d3461-a44b-4895-8d52-9a9bfc0ffb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:25.606324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d3461-a44b-4895-8d52-9a9bfc0ffb90 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:25.606898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d3461-a44b-4895-8d52-9a9bfc0ffb90 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:25.608291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d3461-a44b-4895-8d52-9a9bfc0ffb90 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-7e6d3461-a44b-4895-8d52-9a9bfc0ffb90 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:25.616992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:25.619835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:25.619835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:25.619835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:25.619835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:25.619835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:25.619835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:25.619835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:23:25.619835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:23:25.619835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:23:25.619835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:23:25.619835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:25.619835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:25.619835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:25.619835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:25.619835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:25.619835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:25.620924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:25.620924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:25.620924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:25.620924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:25.620924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:25.620924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:25.620924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:25.620924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:25.620924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:25.620924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:25.620924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:23:25.620924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:23:25.620924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:25.620924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:23:25.620924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 657/1298] 10.222.0.58 () {56 vars in 1117 bytes} [Wed Aug 24 12:23:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:25.624025 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e8f2f-9b61-4cf4-8c8e-a7c1dec83c5d None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:25.642587 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:25.724745 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e8f2f-9b61-4cf4-8c8e-a7c1dec83c5d None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:25.757792 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ba788-79c6-4cc6-a8a1-bc058aa18522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:25.758397 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ba788-79c6-4cc6-a8a1-bc058aa18522 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:25.758679 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ba788-79c6-4cc6-a8a1-bc058aa18522 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:25.758953 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ba788-79c6-4cc6-a8a1-bc058aa18522 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:25.759580 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ba788-79c6-4cc6-a8a1-bc058aa18522 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:25.759917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ba788-79c6-4cc6-a8a1-bc058aa18522 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:25.761283 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ba788-79c6-4cc6-a8a1-bc058aa18522 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:25.766251 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 642/1299] 10.222.0.58 () {56 vars in 1116 bytes} [Wed Aug 24 12:23:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:25.773327 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d3461-a44b-4895-8d52-9a9bfc0ffb90 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:25.804330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:25.873190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d3461-a44b-4895-8d52-9a9bfc0ffb90 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:25.907129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0cac2-9164-420b-be08-42c188b756dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:25.907788 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0cac2-9164-420b-be08-42c188b756dc None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:25.908075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0cac2-9164-420b-be08-42c188b756dc None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:25.908352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0cac2-9164-420b-be08-42c188b756dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:25.909824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0cac2-9164-420b-be08-42c188b756dc None admin] RBAC: Adding query filter params (interface=public, service_id=e935cd2a8f72480d884862d4f7c4197f, region_id=RegionOne) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:25.910131 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0cac2-9164-420b-be08-42c188b756dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:25.911505 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0cac2-9164-420b-be08-42c188b756dc None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:25.916430 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 658/1300] 10.222.0.58 () {56 vars in 1251 bytes} [Wed Aug 24 12:23:25 2022] GET /identity/v3/endpoints?service_id=e935cd2a8f72480d884862d4f7c4197f&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:25.922256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ba788-79c6-4cc6-a8a1-bc058aa18522 None admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:26.001315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:26.019428 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ba788-79c6-4cc6-a8a1-bc058aa18522 None admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:26.052187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9a6dc-7788-4f3a-8cc6-f43dd772835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:26.052782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9a6dc-7788-4f3a-8cc6-f43dd772835c None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:26.053063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9a6dc-7788-4f3a-8cc6-f43dd772835c None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:26.053335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9a6dc-7788-4f3a-8cc6-f43dd772835c None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:26.053933 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9a6dc-7788-4f3a-8cc6-f43dd772835c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:26.055293 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9a6dc-7788-4f3a-8cc6-f43dd772835c None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:26.061952 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 643/1301] 10.222.0.58 () {56 vars in 1087 bytes} [Wed Aug 24 12:23:25 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:23:27.942406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3359cdf-f211-4530-9064-8838a57421fb None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:27.942953 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3359cdf-f211-4530-9064-8838a57421fb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:27.942953 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3359cdf-f211-4530-9064-8838a57421fb None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:27.944362 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 659/1302] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:23:27.949673 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c75fa-b6bc-4944-952d-c321441ff480 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:27.950098 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c75fa-b6bc-4944-952d-c321441ff480 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:27.950225 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c75fa-b6bc-4944-952d-c321441ff480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:27.967854 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-2c4c75fa-b6bc-4944-952d-c321441ff480 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:23:27.969035 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2c4c75fa-b6bc-4944-952d-c321441ff480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:23:28.033488 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 644/1303] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:23:27 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 12:23:28.307496 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3359cdf-f211-4530-9064-8838a57421fb None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:28.402017 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:28.409120 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3359cdf-f211-4530-9064-8838a57421fb None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:28.442046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539d1b4-8b3b-4646-86f4-84dcdad84cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:28.442677 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539d1b4-8b3b-4646-86f4-84dcdad84cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:28.442961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539d1b4-8b3b-4646-86f4-84dcdad84cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:28.443236 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539d1b4-8b3b-4646-86f4-84dcdad84cd7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:28.443845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539d1b4-8b3b-4646-86f4-84dcdad84cd7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:28.445220 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539d1b4-8b3b-4646-86f4-84dcdad84cd7 None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application [None req-8539d1b4-8b3b-4646-86f4-84dcdad84cd7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 12:23:28.454046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 12:23:28.455191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 12:23:28.455191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 12:23:28.455191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 12:23:28.455191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 12:23:28.455191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 12:23:28.455191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 12:23:28.455191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 12:23:28.455191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 12:23:28.455191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 12:23:28.455191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 12:23:28.455191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 12:23:28.455191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 12:23:28.455191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 12:23:28.455191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 12:23:28.455191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 12:23:28.455191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 12:23:28.456654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 12:23:28.456654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application with Lock( Aug 24 12:23:28.456654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 12:23:28.456654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self._enter() Aug 24 12:23:28.456654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 12:23:28.456654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 12:23:28.456654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 12:23:28.456654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application return self.creator() Aug 24 12:23:28.456654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 12:23:28.456654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application created_value = creator( Aug 24 12:23:28.456654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 12:23:28.456654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 12:23:28.456654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 12:23:28.456654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: ERROR keystone.server.flask.application Aug 24 12:23:28.456654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 660/1304] 10.222.0.58 () {56 vars in 1117 bytes} [Wed Aug 24 12:23:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 12:23:28.461654 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c75fa-b6bc-4944-952d-c321441ff480 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:28.537309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:28.559925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c75fa-b6bc-4944-952d-c321441ff480 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:28.593501 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac90ae-a32a-40b1-852c-dadded6824c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:28.594099 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac90ae-a32a-40b1-852c-dadded6824c0 None admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:28.594382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac90ae-a32a-40b1-852c-dadded6824c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:28.594658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac90ae-a32a-40b1-852c-dadded6824c0 None admin] PATH_INFO: `/v3/services` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:28.595282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac90ae-a32a-40b1-852c-dadded6824c0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:23:28.595588 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac90ae-a32a-40b1-852c-dadded6824c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:28.597032 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac90ae-a32a-40b1-852c-dadded6824c0 None admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:28.602007 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 645/1305] 10.222.0.58 () {56 vars in 1116 bytes} [Wed Aug 24 12:23:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:23:28.607637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539d1b4-8b3b-4646-86f4-84dcdad84cd7 None admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:23:28.705234 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539d1b4-8b3b-4646-86f4-84dcdad84cd7 None admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:23:28.738710 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba700287-1933-4d63-b38c-5ec6dd780ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:23:28.739300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba700287-1933-4d63-b38c-5ec6dd780ceb None admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:23:28.739581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba700287-1933-4d63-b38c-5ec6dd780ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:23:28.739881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba700287-1933-4d63-b38c-5ec6dd780ceb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:23:28.740541 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba700287-1933-4d63-b38c-5ec6dd780ceb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:23:28.741902 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba700287-1933-4d63-b38c-5ec6dd780ceb None admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:23:28.761292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 661/1306] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:23:28 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:23:38.599076 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=30 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:23:38.757032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:33.553442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e073a90-94f3-48c2-953d-cd8b3c541631 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:33.554023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e073a90-94f3-48c2-953d-cd8b3c541631 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:33.554371 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e073a90-94f3-48c2-953d-cd8b3c541631 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:33.555949 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 646/1307] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:24:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:33.561363 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8bbed-9841-4ac6-86a0-337686bad82f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:33.561950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8bbed-9841-4ac6-86a0-337686bad82f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:33.562116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f8bbed-9841-4ac6-86a0-337686bad82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:33.662055 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d2f8bbed-9841-4ac6-86a0-337686bad82f None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:33.663542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d2f8bbed-9841-4ac6-86a0-337686bad82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:33.895698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 662/1308] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:24:34.234453 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e33c85-26f8-4eb1-b26c-3e8314f00155 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:34.235026 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e33c85-26f8-4eb1-b26c-3e8314f00155 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:34.235452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e33c85-26f8-4eb1-b26c-3e8314f00155 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:34.237404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 647/1309] 10.222.0.58 () {54 vars in 965 bytes} [Wed Aug 24 12:24:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:34.245028 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094f00f-f1ba-47de-a0c4-8997c24f40a3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:34.245335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094f00f-f1ba-47de-a0c4-8997c24f40a3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:34.245626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094f00f-f1ba-47de-a0c4-8997c24f40a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:34.361999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d094f00f-f1ba-47de-a0c4-8997c24f40a3 None None] MFA Rules not processed for user `34fcb5885325472bbe338cf05d3788f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:34.363303 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d094f00f-f1ba-47de-a0c4-8997c24f40a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:34.390516 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:34.471585 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e33c85-26f8-4eb1-b26c-3e8314f00155 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:34.473385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-89e33c85-26f8-4eb1-b26c-3e8314f00155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:34.481246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 663/1310] 10.222.0.58 () {58 vars in 1062 bytes} [Wed Aug 24 12:24:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:24:34.623746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e33c85-26f8-4eb1-b26c-3e8314f00155 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:34.714731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aabe4f-2eb4-4456-bf19-ebb44c8364bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:34.715996 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aabe4f-2eb4-4456-bf19-ebb44c8364bd service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:34.717132 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aabe4f-2eb4-4456-bf19-ebb44c8364bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:34.717652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aabe4f-2eb4-4456-bf19-ebb44c8364bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:34.719605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-16aabe4f-2eb4-4456-bf19-ebb44c8364bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:34.785031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:34.857418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aabe4f-2eb4-4456-bf19-ebb44c8364bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:34.860354 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aabe4f-2eb4-4456-bf19-ebb44c8364bd service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:34.882160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 648/1311] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:37.112311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42600f2-59c4-4dcd-bd1e-ca5140e786f5 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:37.112913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42600f2-59c4-4dcd-bd1e-ca5140e786f5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:37.112913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42600f2-59c4-4dcd-bd1e-ca5140e786f5 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:37.114221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 664/1312] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:37.119856 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b060bc-c722-40ba-bcd8-4e1fa411d036 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:37.120387 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b060bc-c722-40ba-bcd8-4e1fa411d036 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:37.120387 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b060bc-c722-40ba-bcd8-4e1fa411d036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:37.210982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-f3b060bc-c722-40ba-bcd8-4e1fa411d036 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:37.212170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f3b060bc-c722-40ba-bcd8-4e1fa411d036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:37.308684 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 649/1313] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:37.590296 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42600f2-59c4-4dcd-bd1e-ca5140e786f5 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:37.591865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f42600f2-59c4-4dcd-bd1e-ca5140e786f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:37.696801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42600f2-59c4-4dcd-bd1e-ca5140e786f5 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:37.767841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:37.770405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e759c1-dcb6-4006-b7a3-c34a763f4986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:37.771231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e759c1-dcb6-4006-b7a3-c34a763f4986 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:37.771724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e759c1-dcb6-4006-b7a3-c34a763f4986 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:37.772048 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e759c1-dcb6-4006-b7a3-c34a763f4986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:37.773834 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-67e759c1-dcb6-4006-b7a3-c34a763f4986 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:37.882732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e759c1-dcb6-4006-b7a3-c34a763f4986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:37.883716 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e759c1-dcb6-4006-b7a3-c34a763f4986 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:37.901479 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 665/1314] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:40.115651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f1492-2145-48d5-8eae-7d7e8d0f0812 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:40.116206 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f1492-2145-48d5-8eae-7d7e8d0f0812 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:40.116206 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f1492-2145-48d5-8eae-7d7e8d0f0812 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:40.117612 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 650/1315] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:40.122926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12749f6a-ffe8-4f0b-8188-f7e458e8b234 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:40.123323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12749f6a-ffe8-4f0b-8188-f7e458e8b234 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:40.123469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12749f6a-ffe8-4f0b-8188-f7e458e8b234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:40.193913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-12749f6a-ffe8-4f0b-8188-f7e458e8b234 None None] MFA Rules not processed for user `6c5badd8f28a4d749446523e9d5a6ed3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:40.195042 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-12749f6a-ffe8-4f0b-8188-f7e458e8b234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:40.249789 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:40.296593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 666/1316] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:40 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:40.589144 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f1492-2145-48d5-8eae-7d7e8d0f0812 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:40.591215 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-388f1492-2145-48d5-8eae-7d7e8d0f0812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:40.601714 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:40.696306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f1492-2145-48d5-8eae-7d7e8d0f0812 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:40.768558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471e5a0-8903-4acd-b31a-23de67fa13cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:40.769152 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471e5a0-8903-4acd-b31a-23de67fa13cc service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:40.769426 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471e5a0-8903-4acd-b31a-23de67fa13cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:40.769694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471e5a0-8903-4acd-b31a-23de67fa13cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:40.771275 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-5471e5a0-8903-4acd-b31a-23de67fa13cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:40.832900 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:40.881907 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471e5a0-8903-4acd-b31a-23de67fa13cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:40.883350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5471e5a0-8903-4acd-b31a-23de67fa13cc service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:40.901656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 651/1317] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:40 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:43.348508 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96eff09-996c-4b8c-85a2-921d50dfc257 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:43.349064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96eff09-996c-4b8c-85a2-921d50dfc257 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:43.349064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96eff09-996c-4b8c-85a2-921d50dfc257 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:43.350468 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 667/1318] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:43.355687 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfc5bf-e260-4aba-be61-dc4932446f23 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:43.356112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfc5bf-e260-4aba-be61-dc4932446f23 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:43.356223 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfc5bf-e260-4aba-be61-dc4932446f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:43.421552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-7dbfc5bf-e260-4aba-be61-dc4932446f23 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:43.422933 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7dbfc5bf-e260-4aba-be61-dc4932446f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:43.518850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 652/1319] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:43.802851 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96eff09-996c-4b8c-85a2-921d50dfc257 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:43.804427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f96eff09-996c-4b8c-85a2-921d50dfc257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:43.909113 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96eff09-996c-4b8c-85a2-921d50dfc257 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:43.981349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46a6a0-9cbe-4d42-b519-c30cbd35240e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:43.981956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46a6a0-9cbe-4d42-b519-c30cbd35240e service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:43.982221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46a6a0-9cbe-4d42-b519-c30cbd35240e service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:43.982510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46a6a0-9cbe-4d42-b519-c30cbd35240e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:43.984146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ec46a6a0-9cbe-4d42-b519-c30cbd35240e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:43.991462 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:44.093367 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46a6a0-9cbe-4d42-b519-c30cbd35240e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:44.094799 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46a6a0-9cbe-4d42-b519-c30cbd35240e service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:44.112477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 668/1320] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:46.698041 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01155eb4-96eb-4971-ac17-b86d7d370b29 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:46.698627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01155eb4-96eb-4971-ac17-b86d7d370b29 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:46.698627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01155eb4-96eb-4971-ac17-b86d7d370b29 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:46.699981 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 653/1321] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:46.705735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3199ed2-b080-4749-a596-ccd4c6c678ee None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:46.706170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3199ed2-b080-4749-a596-ccd4c6c678ee None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:46.706280 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3199ed2-b080-4749-a596-ccd4c6c678ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:46.772219 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f3199ed2-b080-4749-a596-ccd4c6c678ee None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:46.773406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f3199ed2-b080-4749-a596-ccd4c6c678ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:46.835155 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:46.876101 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 669/1322] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:47.171246 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01155eb4-96eb-4971-ac17-b86d7d370b29 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:47.172916 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-01155eb4-96eb-4971-ac17-b86d7d370b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:47.193619 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:47.281964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01155eb4-96eb-4971-ac17-b86d7d370b29 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:47.356921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9caa0e-a4e1-4888-927d-05b72a9ca869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:47.357425 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9caa0e-a4e1-4888-927d-05b72a9ca869 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:47.357705 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9caa0e-a4e1-4888-927d-05b72a9ca869 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:47.358041 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9caa0e-a4e1-4888-927d-05b72a9ca869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:47.359832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-bf9caa0e-a4e1-4888-927d-05b72a9ca869 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:47.431007 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:47.471395 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9caa0e-a4e1-4888-927d-05b72a9ca869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:47.472929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9caa0e-a4e1-4888-927d-05b72a9ca869 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:47.490835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 654/1323] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:49.468752 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c33cf-b8c4-4492-870f-cd405b188406 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:49.469218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c33cf-b8c4-4492-870f-cd405b188406 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:49.469300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c33cf-b8c4-4492-870f-cd405b188406 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:49.470825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 670/1324] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:24:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:49.476115 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ec4ba-b377-4527-a876-a99cf5fa2774 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:49.476536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ec4ba-b377-4527-a876-a99cf5fa2774 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:49.476789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ec4ba-b377-4527-a876-a99cf5fa2774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:49.543163 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-665ec4ba-b377-4527-a876-a99cf5fa2774 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:49.544391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-665ec4ba-b377-4527-a876-a99cf5fa2774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:49.642060 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 655/1325] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:49.945114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c33cf-b8c4-4492-870f-cd405b188406 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:49.946633 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0c9c33cf-b8c4-4492-870f-cd405b188406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:50.061681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c33cf-b8c4-4492-870f-cd405b188406 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:50.135472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a13e6-c6fc-4ccd-8da8-ee7c2d49b921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:50.136092 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a13e6-c6fc-4ccd-8da8-ee7c2d49b921 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:50.136380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a13e6-c6fc-4ccd-8da8-ee7c2d49b921 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:50.136658 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a13e6-c6fc-4ccd-8da8-ee7c2d49b921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:50.138259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-625a13e6-c6fc-4ccd-8da8-ee7c2d49b921 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:50.156975 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:50.251177 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a13e6-c6fc-4ccd-8da8-ee7c2d49b921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:50.252642 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a13e6-c6fc-4ccd-8da8-ee7c2d49b921 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:50.273967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 671/1326] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:52.262957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d7441-e275-4f31-9a4a-6c9c23830bbb None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:52.263511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d7441-e275-4f31-9a4a-6c9c23830bbb None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:52.263511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d7441-e275-4f31-9a4a-6c9c23830bbb None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:52.265768 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 656/1327] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:24:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:52.271649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302196e-f082-4c8b-8ce8-17b8a5e3bb6a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:52.272203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302196e-f082-4c8b-8ce8-17b8a5e3bb6a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:52.272203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302196e-f082-4c8b-8ce8-17b8a5e3bb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:52.340463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e302196e-f082-4c8b-8ce8-17b8a5e3bb6a None None] MFA Rules not processed for user `6c5badd8f28a4d749446523e9d5a6ed3`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:52.341663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e302196e-f082-4c8b-8ce8-17b8a5e3bb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:52.406255 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:52.439020 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 672/1328] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:52 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:52.724627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d7441-e275-4f31-9a4a-6c9c23830bbb None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:52.726193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-730d7441-e275-4f31-9a4a-6c9c23830bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:52.758878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:52.835741 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d7441-e275-4f31-9a4a-6c9c23830bbb None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:52.910449 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82cae1f-4286-462b-85d4-71b595432c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:52.910994 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cae1f-4286-462b-85d4-71b595432c0c service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:52.911274 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cae1f-4286-462b-85d4-71b595432c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:52.911544 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82cae1f-4286-462b-85d4-71b595432c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:52.913169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e82cae1f-4286-462b-85d4-71b595432c0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:52.991532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:53.024540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82cae1f-4286-462b-85d4-71b595432c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:53.025980 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82cae1f-4286-462b-85d4-71b595432c0c service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:53.045242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 657/1329] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:52 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:55.221469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31238d9-4cc1-4b52-a286-f580e7b0c880 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:55.221994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31238d9-4cc1-4b52-a286-f580e7b0c880 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:55.221994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31238d9-4cc1-4b52-a286-f580e7b0c880 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:55.223297 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 673/1330] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:55.228467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f44918-79b1-4a8b-857d-7f65017e1f94 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:55.228892 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f44918-79b1-4a8b-857d-7f65017e1f94 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:55.229024 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f44918-79b1-4a8b-857d-7f65017e1f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:55.293752 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-27f44918-79b1-4a8b-857d-7f65017e1f94 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:55.294964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-27f44918-79b1-4a8b-857d-7f65017e1f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:55.390607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 658/1331] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:55.677857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31238d9-4cc1-4b52-a286-f580e7b0c880 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:55.679973 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f31238d9-4cc1-4b52-a286-f580e7b0c880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:55.784347 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31238d9-4cc1-4b52-a286-f580e7b0c880 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:55.859013 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8067e-19d2-47ea-9916-35f577697706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:55.859992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8067e-19d2-47ea-9916-35f577697706 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:55.859992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8067e-19d2-47ea-9916-35f577697706 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:55.860171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8067e-19d2-47ea-9916-35f577697706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:55.861768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-55b8067e-19d2-47ea-9916-35f577697706 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:55.889688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:55.972303 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8067e-19d2-47ea-9916-35f577697706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:55.973730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8067e-19d2-47ea-9916-35f577697706 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:55.991512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 674/1332] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:24:57.969690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720952e3-e2f8-43c1-bcff-3fbfcf4cd771 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:57.970217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720952e3-e2f8-43c1-bcff-3fbfcf4cd771 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:57.970217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720952e3-e2f8-43c1-bcff-3fbfcf4cd771 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:57.971601 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 659/1333] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:24:57.976987 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a05068-d95f-4e81-a862-59dd9cf06b57 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:57.977399 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a05068-d95f-4e81-a862-59dd9cf06b57 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:57.977539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a05068-d95f-4e81-a862-59dd9cf06b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:58.044178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f4a05068-d95f-4e81-a862-59dd9cf06b57 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:24:58.045974 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f4a05068-d95f-4e81-a862-59dd9cf06b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:58.118690 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:58.142257 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 675/1334] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:24:58.428092 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720952e3-e2f8-43c1-bcff-3fbfcf4cd771 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:24:58.429632 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-720952e3-e2f8-43c1-bcff-3fbfcf4cd771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:58.470196 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:58.534010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720952e3-e2f8-43c1-bcff-3fbfcf4cd771 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:24:58.606097 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c053e7-5cdc-469e-82ba-752e191682c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:24:58.606728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c053e7-5cdc-469e-82ba-752e191682c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:24:58.607012 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c053e7-5cdc-469e-82ba-752e191682c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:24:58.607285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c053e7-5cdc-469e-82ba-752e191682c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:24:58.609812 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-40c053e7-5cdc-469e-82ba-752e191682c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:24:58.691929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:24:58.717490 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c053e7-5cdc-469e-82ba-752e191682c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:24:58.718918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c053e7-5cdc-469e-82ba-752e191682c6 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:24:58.736873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 660/1335] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:00.955592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ae012-b02c-4117-9e1b-23352c611fbb None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:00.956163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ae012-b02c-4117-9e1b-23352c611fbb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:00.956163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ae012-b02c-4117-9e1b-23352c611fbb None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:00.957502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 676/1336] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:00.962677 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a8681-39a3-43ec-bdbe-b888cdbd1d58 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:00.963079 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a8681-39a3-43ec-bdbe-b888cdbd1d58 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:00.963210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a8681-39a3-43ec-bdbe-b888cdbd1d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:01.028979 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-c01a8681-39a3-43ec-bdbe-b888cdbd1d58 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:01.030821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c01a8681-39a3-43ec-bdbe-b888cdbd1d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:01.124998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 661/1337] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:25:01.422693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c0a0f-cf95-4ee3-85cc-97d36f731351 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:01.422972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c0a0f-cf95-4ee3-85cc-97d36f731351 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:01.423245 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c0a0f-cf95-4ee3-85cc-97d36f731351 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:01.424695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 677/1338] 10.222.0.58 () {54 vars in 965 bytes} [Wed Aug 24 12:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:01.430627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b4dda-7f4a-4f69-8210-03b5ca823c04 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:01.430918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b4dda-7f4a-4f69-8210-03b5ca823c04 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:01.431194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b4dda-7f4a-4f69-8210-03b5ca823c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:01.486017 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:01.507576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-199b4dda-7f4a-4f69-8210-03b5ca823c04 None None] MFA Rules not processed for user `34fcb5885325472bbe338cf05d3788f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:01.508818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-199b4dda-7f4a-4f69-8210-03b5ca823c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:01.584688 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 662/1339] 10.222.0.58 () {58 vars in 1062 bytes} [Wed Aug 24 12:25:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:01.594372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c0a0f-cf95-4ee3-85cc-97d36f731351 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:01.595975 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e18c0a0f-cf95-4ee3-85cc-97d36f731351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:01.698236 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c0a0f-cf95-4ee3-85cc-97d36f731351 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:01.770395 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38845783-cc3c-4b6f-b715-221743b0013d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:01.770993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38845783-cc3c-4b6f-b715-221743b0013d service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:01.771271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38845783-cc3c-4b6f-b715-221743b0013d service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:01.771539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38845783-cc3c-4b6f-b715-221743b0013d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:01.773144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-38845783-cc3c-4b6f-b715-221743b0013d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:01.808282 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:01.897290 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38845783-cc3c-4b6f-b715-221743b0013d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:01.898765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38845783-cc3c-4b6f-b715-221743b0013d service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:01.917513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 678/1340] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:06.385142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019f0d9-b4bd-4e58-9478-946ecc175df6 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:06.385615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019f0d9-b4bd-4e58-9478-946ecc175df6 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:06.385615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019f0d9-b4bd-4e58-9478-946ecc175df6 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:06.387058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 663/1341] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:06.393533 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16fcf0-f1e8-4d27-9858-ccca27a1d8fe None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:06.393955 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16fcf0-f1e8-4d27-9858-ccca27a1d8fe None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:06.394229 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16fcf0-f1e8-4d27-9858-ccca27a1d8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:06.463485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-bc16fcf0-f1e8-4d27-9858-ccca27a1d8fe None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:06.464724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bc16fcf0-f1e8-4d27-9858-ccca27a1d8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:06.555579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:06.567559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 679/1342] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:06.853367 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019f0d9-b4bd-4e58-9478-946ecc175df6 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:06.854883 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d019f0d9-b4bd-4e58-9478-946ecc175df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:06.973111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019f0d9-b4bd-4e58-9478-946ecc175df6 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:07.013340 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:07.050338 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b50e1e-9d08-4977-bac8-e8f04d32ad8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:07.050947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b50e1e-9d08-4977-bac8-e8f04d32ad8b service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:07.051228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b50e1e-9d08-4977-bac8-e8f04d32ad8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:07.051501 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b50e1e-9d08-4977-bac8-e8f04d32ad8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:07.053130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-77b50e1e-9d08-4977-bac8-e8f04d32ad8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:07.167936 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b50e1e-9d08-4977-bac8-e8f04d32ad8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:07.169298 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b50e1e-9d08-4977-bac8-e8f04d32ad8b service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:07.188183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 664/1343] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:09.781761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb0d13-7e13-43a4-8a0d-f62a35c02d19 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:09.782287 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb0d13-7e13-43a4-8a0d-f62a35c02d19 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:09.782287 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb0d13-7e13-43a4-8a0d-f62a35c02d19 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:09.783602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 680/1344] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:09.789376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c779c8-3d6d-4a57-9b3c-27d93ad36222 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:09.789801 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c779c8-3d6d-4a57-9b3c-27d93ad36222 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:09.790006 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c779c8-3d6d-4a57-9b3c-27d93ad36222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:09.860729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-15c779c8-3d6d-4a57-9b3c-27d93ad36222 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:09.861934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-15c779c8-3d6d-4a57-9b3c-27d93ad36222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:09.884977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:09.972915 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 665/1345] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:10.256742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb0d13-7e13-43a4-8a0d-f62a35c02d19 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:10.258413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6deb0d13-7e13-43a4-8a0d-f62a35c02d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:10.365002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb0d13-7e13-43a4-8a0d-f62a35c02d19 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:10.439344 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00032678-d79a-4b8d-9483-a70e587822ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:10.439977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00032678-d79a-4b8d-9483-a70e587822ef service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:10.440255 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00032678-d79a-4b8d-9483-a70e587822ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:10.440587 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00032678-d79a-4b8d-9483-a70e587822ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:10.442180 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-00032678-d79a-4b8d-9483-a70e587822ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:10.489743 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:10.559724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00032678-d79a-4b8d-9483-a70e587822ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:10.561785 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00032678-d79a-4b8d-9483-a70e587822ef service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:10.580956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 681/1346] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:14.792339 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fdbd7-1fa2-4822-ac21-064d796794dd None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:14.792871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fdbd7-1fa2-4822-ac21-064d796794dd None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:14.792871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fdbd7-1fa2-4822-ac21-064d796794dd None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:14.794257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 666/1347] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:14.800064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff27485-6d97-4a78-87ef-0541494703e6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:14.800509 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff27485-6d97-4a78-87ef-0541494703e6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:14.800606 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff27485-6d97-4a78-87ef-0541494703e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:14.874572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-cff27485-6d97-4a78-87ef-0541494703e6 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:14.875862 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cff27485-6d97-4a78-87ef-0541494703e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:14.976798 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:14.979605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 682/1348] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:15.271166 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fdbd7-1fa2-4822-ac21-064d796794dd None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:15.272731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-0d0fdbd7-1fa2-4822-ac21-064d796794dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:15.380763 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fdbd7-1fa2-4822-ac21-064d796794dd None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:15.427114 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:15.453965 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6099ef-6ba4-493f-9b3e-fb7c7aa410b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:15.454567 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6099ef-6ba4-493f-9b3e-fb7c7aa410b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:15.454848 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6099ef-6ba4-493f-9b3e-fb7c7aa410b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:15.455122 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6099ef-6ba4-493f-9b3e-fb7c7aa410b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:15.456747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-eb6099ef-6ba4-493f-9b3e-fb7c7aa410b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:15.572415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6099ef-6ba4-493f-9b3e-fb7c7aa410b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:15.573896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6099ef-6ba4-493f-9b3e-fb7c7aa410b5 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:15.593369 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 667/1349] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:20.525257 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ba8f0-c4e7-4f19-aa55-e1f92414a8b4 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:20.525701 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ba8f0-c4e7-4f19-aa55-e1f92414a8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:20.525783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ba8f0-c4e7-4f19-aa55-e1f92414a8b4 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:20.527557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 683/1350] 10.222.0.58 () {54 vars in 921 bytes} [Wed Aug 24 12:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:20.533846 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7c371-91e7-42e6-aed1-a233dfa4ec83 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:20.534274 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7c371-91e7-42e6-aed1-a233dfa4ec83 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:20.534818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7c371-91e7-42e6-aed1-a233dfa4ec83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:20.628998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-edf7c371-91e7-42e6-aed1-a233dfa4ec83 None None] MFA Rules not processed for user `b7ba45e3cff74962965b248f9f017f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:20.630791 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-edf7c371-91e7-42e6-aed1-a233dfa4ec83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:20.650439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:20.773385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 668/1351] 10.222.0.58 () {58 vars in 1018 bytes} [Wed Aug 24 12:25:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:24.947068 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6a0d4-f506-4361-bede-66a5b296645c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:24.947614 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6a0d4-f506-4361-bede-66a5b296645c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:24.947849 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6a0d4-f506-4361-bede-66a5b296645c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:24.949389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 684/1352] 10.222.0.58 () {54 vars in 921 bytes} [Wed Aug 24 12:25:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:24.955585 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f3f1c-a23f-431d-bf83-3b64d5b71418 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:24.956078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f3f1c-a23f-431d-bf83-3b64d5b71418 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:24.956161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f3f1c-a23f-431d-bf83-3b64d5b71418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:25.035864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-ee4f3f1c-a23f-431d-bf83-3b64d5b71418 None None] MFA Rules not processed for user `b7ba45e3cff74962965b248f9f017f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:25.037064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-ee4f3f1c-a23f-431d-bf83-3b64d5b71418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:25.114984 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:25.154599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 669/1353] 10.222.0.58 () {58 vars in 1018 bytes} [Wed Aug 24 12:25:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:26.712715 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb075a-84d2-4304-9f47-7859c0534bad None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:26.713257 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb075a-84d2-4304-9f47-7859c0534bad None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:26.713257 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb075a-84d2-4304-9f47-7859c0534bad None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:26.714621 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 685/1354] 10.222.0.58 () {54 vars in 921 bytes} [Wed Aug 24 12:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:26.720674 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317a102-e3e0-4875-8096-7db785a1d89e None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:26.721143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317a102-e3e0-4875-8096-7db785a1d89e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:26.721250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317a102-e3e0-4875-8096-7db785a1d89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:26.799130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-d317a102-e3e0-4875-8096-7db785a1d89e None None] MFA Rules not processed for user `b7ba45e3cff74962965b248f9f017f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:26.800377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d317a102-e3e0-4875-8096-7db785a1d89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:26.910627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 670/1355] 10.222.0.58 () {58 vars in 1018 bytes} [Wed Aug 24 12:25:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:25:30.335221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e48d94-6748-4947-9e6b-2761bd3daf10 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:30.335810 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e48d94-6748-4947-9e6b-2761bd3daf10 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:30.335810 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e48d94-6748-4947-9e6b-2761bd3daf10 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:30.337171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 686/1356] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:25:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:30.344815 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be4cba-6ef1-415f-b1df-b4959e81a34c None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:30.345260 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be4cba-6ef1-415f-b1df-b4959e81a34c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:30.345385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be4cba-6ef1-415f-b1df-b4959e81a34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:30.381411 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:30.431793 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-50be4cba-6ef1-415f-b1df-b4959e81a34c None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:30.432987 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-50be4cba-6ef1-415f-b1df-b4959e81a34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:30.553622 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 671/1357] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:30.864250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1effb-86cd-402b-bf3e-5998057a34d9 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:30.864681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1effb-86cd-402b-bf3e-5998057a34d9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:30.865059 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1effb-86cd-402b-bf3e-5998057a34d9 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:30.867733 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 687/1358] 10.222.0.58 () {54 vars in 962 bytes} [Wed Aug 24 12:25:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:30.882218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b67113-3588-47f7-bd61-af69d1083528 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:30.882532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b67113-3588-47f7-bd61-af69d1083528 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:30.882763 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b67113-3588-47f7-bd61-af69d1083528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:30.972316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-b8b67113-3588-47f7-bd61-af69d1083528 None None] MFA Rules not processed for user `311dbed3ab0b44a7830a93227ff9ad33`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:30.973811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b8b67113-3588-47f7-bd61-af69d1083528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:31.025934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:31.099058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 672/1359] 10.222.0.58 () {58 vars in 1059 bytes} [Wed Aug 24 12:25:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:31.107483 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1effb-86cd-402b-bf3e-5998057a34d9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:31.109096 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8bd1effb-86cd-402b-bf3e-5998057a34d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:31.270469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1effb-86cd-402b-bf3e-5998057a34d9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:31.348217 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:31.392187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c5ab5-020d-4120-a9a1-84888a628491 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:31.393010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c5ab5-020d-4120-a9a1-84888a628491 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:31.393438 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c5ab5-020d-4120-a9a1-84888a628491 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:31.393868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c5ab5-020d-4120-a9a1-84888a628491 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:31.395641 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a51c5ab5-020d-4120-a9a1-84888a628491 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:31.495972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c5ab5-020d-4120-a9a1-84888a628491 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:31.498738 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c5ab5-020d-4120-a9a1-84888a628491 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:31.519304 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 688/1360] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:33.686583 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a586b-4efe-42cc-8506-433ed96e671f None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:33.687142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a586b-4efe-42cc-8506-433ed96e671f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:33.687308 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a586b-4efe-42cc-8506-433ed96e671f None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:33.688965 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 673/1361] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:25:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:33.694268 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57721863-908d-4d83-8a98-3e70ad6ceb6d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:33.694688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57721863-908d-4d83-8a98-3e70ad6ceb6d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:33.694825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57721863-908d-4d83-8a98-3e70ad6ceb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:33.767213 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-57721863-908d-4d83-8a98-3e70ad6ceb6d None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:33.768612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-57721863-908d-4d83-8a98-3e70ad6ceb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:33.810661 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:33.885981 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 689/1362] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:34.189759 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a586b-4efe-42cc-8506-433ed96e671f None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:34.191384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-364a586b-4efe-42cc-8506-433ed96e671f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:34.360298 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a586b-4efe-42cc-8506-433ed96e671f None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:34.370886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:34.507058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da53ee1-2e4b-4295-b366-07a1fab5d03c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:34.507764 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da53ee1-2e4b-4295-b366-07a1fab5d03c service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:34.508263 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da53ee1-2e4b-4295-b366-07a1fab5d03c service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:34.508772 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da53ee1-2e4b-4295-b366-07a1fab5d03c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:34.511023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-8da53ee1-2e4b-4295-b366-07a1fab5d03c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:34.632605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da53ee1-2e4b-4295-b366-07a1fab5d03c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:34.634321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da53ee1-2e4b-4295-b366-07a1fab5d03c service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:34.655973 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 674/1363] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:37.052178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7728bb0-8bd7-4399-8a4b-c6f41f056c47 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:37.052712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7728bb0-8bd7-4399-8a4b-c6f41f056c47 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:37.052712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7728bb0-8bd7-4399-8a4b-c6f41f056c47 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:37.054085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 690/1364] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:25:37.059684 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91684f-8080-4bb3-89f1-958f70ba65e0 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:37.060287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91684f-8080-4bb3-89f1-958f70ba65e0 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:37.060392 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91684f-8080-4bb3-89f1-958f70ba65e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:37.077561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:37.133809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-5c91684f-8080-4bb3-89f1-958f70ba65e0 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:37.135180 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-5c91684f-8080-4bb3-89f1-958f70ba65e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:37.239758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 675/1365] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:37.531475 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd70aad-a504-4271-ab76-fc488bc8a79d None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:37.531803 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd70aad-a504-4271-ab76-fc488bc8a79d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:37.532079 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd70aad-a504-4271-ab76-fc488bc8a79d None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:37.533570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 691/1366] 10.222.0.58 () {54 vars in 962 bytes} [Wed Aug 24 12:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:37.538911 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b43b62-72d9-4e4d-a333-d63a508976cd None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:37.539204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b43b62-72d9-4e4d-a333-d63a508976cd None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:37.539481 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b43b62-72d9-4e4d-a333-d63a508976cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:37.622085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-e8b43b62-72d9-4e4d-a333-d63a508976cd None None] MFA Rules not processed for user `311dbed3ab0b44a7830a93227ff9ad33`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:37.623879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e8b43b62-72d9-4e4d-a333-d63a508976cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:37.663074 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:37.756653 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 676/1367] 10.222.0.58 () {58 vars in 1059 bytes} [Wed Aug 24 12:25:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:37.765418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd70aad-a504-4271-ab76-fc488bc8a79d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:37.767074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0cd70aad-a504-4271-ab76-fc488bc8a79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:37.937542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd70aad-a504-4271-ab76-fc488bc8a79d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:37.959355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:38.057855 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f08b7-24bc-4805-a739-41e47ef74767 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:38.058618 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f08b7-24bc-4805-a739-41e47ef74767 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:38.059000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f08b7-24bc-4805-a739-41e47ef74767 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:38.059946 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f08b7-24bc-4805-a739-41e47ef74767 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:38.061760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-090f08b7-24bc-4805-a739-41e47ef74767 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:38.156988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f08b7-24bc-4805-a739-41e47ef74767 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:38.158787 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f08b7-24bc-4805-a739-41e47ef74767 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:38.179914 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 692/1368] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:38.735972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70eb5c-28a4-45fb-9e7a-f59c0145e4e1 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:38.736558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70eb5c-28a4-45fb-9e7a-f59c0145e4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:38.736558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d70eb5c-28a4-45fb-9e7a-f59c0145e4e1 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:38.738034 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 677/1369] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:38.743221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ac571-648c-45a6-a78a-8a9d9e1f6e21 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:38.743513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ac571-648c-45a6-a78a-8a9d9e1f6e21 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:38.743849 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ac571-648c-45a6-a78a-8a9d9e1f6e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:38.776936 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:38.817503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-033ac571-648c-45a6-a78a-8a9d9e1f6e21 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:38.819498 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-033ac571-648c-45a6-a78a-8a9d9e1f6e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:38.930554 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 693/1370] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:39.529989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad7407-ba58-4432-a494-1c1eaa94f729 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:39.529989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad7407-ba58-4432-a494-1c1eaa94f729 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:39.529989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad7407-ba58-4432-a494-1c1eaa94f729 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:39.531722 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 678/1371] 10.222.0.58 () {54 vars in 964 bytes} [Wed Aug 24 12:25:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:39.536055 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289f8d3-d7fc-46b1-9cc2-bc135bf7a560 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:39.536647 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289f8d3-d7fc-46b1-9cc2-bc135bf7a560 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:39.536647 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289f8d3-d7fc-46b1-9cc2-bc135bf7a560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:39.614457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6289f8d3-d7fc-46b1-9cc2-bc135bf7a560 None None] MFA Rules not processed for user `a42d8e462ba542a5bf58ba052dceeb8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:39.615694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6289f8d3-d7fc-46b1-9cc2-bc135bf7a560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:39.665143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:39.702907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 694/1372] 10.222.0.58 () {58 vars in 1061 bytes} [Wed Aug 24 12:25:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:39.710126 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad7407-ba58-4432-a494-1c1eaa94f729 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:39.712088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c9ad7407-ba58-4432-a494-1c1eaa94f729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:39.833891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad7407-ba58-4432-a494-1c1eaa94f729 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:39.886304 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:39.914139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2984e4-cccb-4e57-b941-6cbffdafd347 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:39.914758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2984e4-cccb-4e57-b941-6cbffdafd347 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:39.915084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2984e4-cccb-4e57-b941-6cbffdafd347 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:39.915401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2984e4-cccb-4e57-b941-6cbffdafd347 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:39.917263 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-0c2984e4-cccb-4e57-b941-6cbffdafd347 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:40.051394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2984e4-cccb-4e57-b941-6cbffdafd347 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:40.053214 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2984e4-cccb-4e57-b941-6cbffdafd347 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:40.084774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 679/1373] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:25:40.214299 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f52a65-791e-49ff-82fd-c693aea3251e None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:40.214299 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f52a65-791e-49ff-82fd-c693aea3251e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:40.214617 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f52a65-791e-49ff-82fd-c693aea3251e None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:40.216639 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 695/1374] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:25:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:40.225101 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c10406-bdb6-4077-9ecb-95c16c317cc0 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:40.225557 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c10406-bdb6-4077-9ecb-95c16c317cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:40.225865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c10406-bdb6-4077-9ecb-95c16c317cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:40.311807 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-53c10406-bdb6-4077-9ecb-95c16c317cc0 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:40.313033 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-53c10406-bdb6-4077-9ecb-95c16c317cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:40.335193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:40.413029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 680/1375] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:40.703518 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f52a65-791e-49ff-82fd-c693aea3251e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:40.705082 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-57f52a65-791e-49ff-82fd-c693aea3251e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:40.859413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f52a65-791e-49ff-82fd-c693aea3251e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:40.915782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:40.979529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c36ae-fb99-4c9c-9d69-3a5da884bc91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:40.980105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c36ae-fb99-4c9c-9d69-3a5da884bc91 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:40.980387 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c36ae-fb99-4c9c-9d69-3a5da884bc91 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:40.980681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c36ae-fb99-4c9c-9d69-3a5da884bc91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:40.982532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-302c36ae-fb99-4c9c-9d69-3a5da884bc91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:41.072271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c36ae-fb99-4c9c-9d69-3a5da884bc91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:41.073863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c36ae-fb99-4c9c-9d69-3a5da884bc91 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:41.092632 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 696/1376] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:43.071519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d56054-7730-46f4-8ee5-22ea85ee0a7e None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:43.072080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d56054-7730-46f4-8ee5-22ea85ee0a7e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:43.072080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d56054-7730-46f4-8ee5-22ea85ee0a7e None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:43.073474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 681/1377] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:25:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:43.078774 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65a7b2-f8ac-40dc-b72f-82d3d08725b9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:43.079201 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65a7b2-f8ac-40dc-b72f-82d3d08725b9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:43.079467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe65a7b2-f8ac-40dc-b72f-82d3d08725b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:43.160311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fe65a7b2-f8ac-40dc-b72f-82d3d08725b9 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:43.161458 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fe65a7b2-f8ac-40dc-b72f-82d3d08725b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:43.169344 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:43.274585 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 697/1378] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:43.570190 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d56054-7730-46f4-8ee5-22ea85ee0a7e None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:43.571986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a3d56054-7730-46f4-8ee5-22ea85ee0a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:43.760317 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d56054-7730-46f4-8ee5-22ea85ee0a7e None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:43.764423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:43.896826 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80295b68-5165-459a-8e3f-cd2f418e3109 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:43.897446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80295b68-5165-459a-8e3f-cd2f418e3109 service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:43.897729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80295b68-5165-459a-8e3f-cd2f418e3109 service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:43.898002 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80295b68-5165-459a-8e3f-cd2f418e3109 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:43.899904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-80295b68-5165-459a-8e3f-cd2f418e3109 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:43.998717 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80295b68-5165-459a-8e3f-cd2f418e3109 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:44.000513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80295b68-5165-459a-8e3f-cd2f418e3109 service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:44.019742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 682/1379] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:45.982812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1cc3c-075c-4a81-857a-aa0db483f29e None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:45.983352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1cc3c-075c-4a81-857a-aa0db483f29e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:45.983352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1cc3c-075c-4a81-857a-aa0db483f29e None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:45.985041 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 698/1380] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:25:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:45.990116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109395de-13e1-4b14-8db7-9bda802c7d27 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:45.990504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109395de-13e1-4b14-8db7-9bda802c7d27 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:45.990691 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109395de-13e1-4b14-8db7-9bda802c7d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:46.002798 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:46.068694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-109395de-13e1-4b14-8db7-9bda802c7d27 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:46.069905 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-109395de-13e1-4b14-8db7-9bda802c7d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:46.175335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 683/1381] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:46.473806 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1cc3c-075c-4a81-857a-aa0db483f29e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:46.475395 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f2b1cc3c-075c-4a81-857a-aa0db483f29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:46.639442 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:46.652133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1cc3c-075c-4a81-857a-aa0db483f29e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:46.778888 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935c256-25ce-4f76-b377-cb8bb70f3ce9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:46.779481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935c256-25ce-4f76-b377-cb8bb70f3ce9 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:46.779795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935c256-25ce-4f76-b377-cb8bb70f3ce9 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:46.780073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935c256-25ce-4f76-b377-cb8bb70f3ce9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:46.781693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1935c256-25ce-4f76-b377-cb8bb70f3ce9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:46.886770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935c256-25ce-4f76-b377-cb8bb70f3ce9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:46.889162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935c256-25ce-4f76-b377-cb8bb70f3ce9 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:46.900965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:46.911462 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 699/1382] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:49.091927 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7514019-f4bf-4a4d-a85e-3c1b5102d3c1 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:49.092705 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7514019-f4bf-4a4d-a85e-3c1b5102d3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:49.092705 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7514019-f4bf-4a4d-a85e-3c1b5102d3c1 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:49.093960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 684/1383] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:25:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:49.100073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08186a49-2839-4b81-8cb4-2759fd4dc72e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:49.100724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08186a49-2839-4b81-8cb4-2759fd4dc72e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:49.100905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08186a49-2839-4b81-8cb4-2759fd4dc72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:49.185642 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-08186a49-2839-4b81-8cb4-2759fd4dc72e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:49.186845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-08186a49-2839-4b81-8cb4-2759fd4dc72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:49.288186 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 700/1384] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:49.588329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7514019-f4bf-4a4d-a85e-3c1b5102d3c1 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:49.589988 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f7514019-f4bf-4a4d-a85e-3c1b5102d3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:49.686705 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:49.753478 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7514019-f4bf-4a4d-a85e-3c1b5102d3c1 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:49.872069 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa1333-8704-4df2-a0f7-3b0008d765e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:49.872852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa1333-8704-4df2-a0f7-3b0008d765e0 service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:49.873382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa1333-8704-4df2-a0f7-3b0008d765e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:49.873750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa1333-8704-4df2-a0f7-3b0008d765e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:49.876456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-92aa1333-8704-4df2-a0f7-3b0008d765e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:49.941575 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:49.985449 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa1333-8704-4df2-a0f7-3b0008d765e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:49.987078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa1333-8704-4df2-a0f7-3b0008d765e0 service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:50.006254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 685/1385] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:52.038386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecdd75-531b-4488-9e8f-bc94e1e6c45e None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:52.038961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecdd75-531b-4488-9e8f-bc94e1e6c45e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:52.038961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecdd75-531b-4488-9e8f-bc94e1e6c45e None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:52.040331 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 701/1386] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:52.046583 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b06b21-35c3-4332-9551-76b020a2b7e1 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:52.047070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b06b21-35c3-4332-9551-76b020a2b7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:52.047250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b06b21-35c3-4332-9551-76b020a2b7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:52.117277 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-19b06b21-35c3-4332-9551-76b020a2b7e1 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:52.118570 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-19b06b21-35c3-4332-9551-76b020a2b7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:52.223269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 686/1387] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:52.529767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecdd75-531b-4488-9e8f-bc94e1e6c45e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:52.531289 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-82ecdd75-531b-4488-9e8f-bc94e1e6c45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:52.620816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:52.694567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecdd75-531b-4488-9e8f-bc94e1e6c45e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:52.814335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81b534-b7e0-43f6-8c79-1e02dfe54307 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:52.815005 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81b534-b7e0-43f6-8c79-1e02dfe54307 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:52.815227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81b534-b7e0-43f6-8c79-1e02dfe54307 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:52.815500 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81b534-b7e0-43f6-8c79-1e02dfe54307 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:52.817143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4b81b534-b7e0-43f6-8c79-1e02dfe54307 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:52.861557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:52.931776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81b534-b7e0-43f6-8c79-1e02dfe54307 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:52.934695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81b534-b7e0-43f6-8c79-1e02dfe54307 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:52.959700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 702/1388] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:54.938247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979a57b-1577-4b40-848d-db2866073e97 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:54.938850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979a57b-1577-4b40-848d-db2866073e97 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:54.938850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979a57b-1577-4b40-848d-db2866073e97 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:54.940188 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 687/1389] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:25:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:54.945931 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765da99c-0394-4283-ad21-8f5e567d81ae None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:54.946374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765da99c-0394-4283-ad21-8f5e567d81ae None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:54.946478 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765da99c-0394-4283-ad21-8f5e567d81ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:55.013238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-765da99c-0394-4283-ad21-8f5e567d81ae None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:55.014543 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-765da99c-0394-4283-ad21-8f5e567d81ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:55.116549 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 703/1390] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:55.282557 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6e0ed-eeae-4aff-aef8-cc2880619df8 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:55.282810 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6e0ed-eeae-4aff-aef8-cc2880619df8 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:55.283095 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6e0ed-eeae-4aff-aef8-cc2880619df8 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:55.284511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 688/1391] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:55.289852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa26b9-a56e-44b3-a9a9-cc4b96439a98 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:55.290144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa26b9-a56e-44b3-a9a9-cc4b96439a98 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:55.290422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa26b9-a56e-44b3-a9a9-cc4b96439a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:55.306600 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:55.358788 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-cdaa26b9-a56e-44b3-a9a9-cc4b96439a98 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:55.360045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cdaa26b9-a56e-44b3-a9a9-cc4b96439a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:55.409149 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae6e0ed-eeae-4aff-aef8-cc2880619df8 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:55.411001 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-eae6e0ed-eeae-4aff-aef8-cc2880619df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:55.443315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:55.458712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 704/1392] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:55.563333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae6e0ed-eeae-4aff-aef8-cc2880619df8 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:55.667022 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:55.679921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46c468-8a99-4744-93d0-8fcdfc6ca455 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:55.680535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46c468-8a99-4744-93d0-8fcdfc6ca455 service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:55.680812 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46c468-8a99-4744-93d0-8fcdfc6ca455 service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:55.681083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46c468-8a99-4744-93d0-8fcdfc6ca455 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:55.683090 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-ad46c468-8a99-4744-93d0-8fcdfc6ca455 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:55.763379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a59566-1a1b-4795-a95a-78a37038bd52 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:55.763701 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a59566-1a1b-4795-a95a-78a37038bd52 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:55.763978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a59566-1a1b-4795-a95a-78a37038bd52 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:55.765365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 705/1393] 10.222.0.58 () {54 vars in 967 bytes} [Wed Aug 24 12:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:55.770596 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e1926-0371-46e7-aac3-a74141720eac None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:55.770881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e1926-0371-46e7-aac3-a74141720eac None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:55.771156 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e1926-0371-46e7-aac3-a74141720eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:55.781227 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46c468-8a99-4744-93d0-8fcdfc6ca455 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:55.782822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46c468-8a99-4744-93d0-8fcdfc6ca455 service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:55.800996 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 689/1394] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:25:55.851669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2f1e1926-0371-46e7-aac3-a74141720eac None None] MFA Rules not processed for user `dd8df2f3357d4660bc22366f9746f89c`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:55.852934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2f1e1926-0371-46e7-aac3-a74141720eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:55.894683 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:55.944560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 706/1395] 10.222.0.58 () {58 vars in 1064 bytes} [Wed Aug 24 12:25:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:25:55.953606 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46c468-8a99-4744-93d0-8fcdfc6ca455 service nova] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:55.954867 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-ad46c468-8a99-4744-93d0-8fcdfc6ca455 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:56.038781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46c468-8a99-4744-93d0-8fcdfc6ca455 service nova] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:56.080748 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:56.114738 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd886992-eabc-4bd4-9fd2-0443a66b8b47 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd8df2f3357d4660bc22366f9746f89c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:56.115351 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd886992-eabc-4bd4-9fd2-0443a66b8b47 service glance] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:56.115652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd886992-eabc-4bd4-9fd2-0443a66b8b47 service glance] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:56.115944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd886992-eabc-4bd4-9fd2-0443a66b8b47 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:56.117533 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-cd886992-eabc-4bd4-9fd2-0443a66b8b47 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:56.227359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd886992-eabc-4bd4-9fd2-0443a66b8b47 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:56.228836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd886992-eabc-4bd4-9fd2-0443a66b8b47 service glance] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:56.247276 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 690/1396] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:25:56.998147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640ea4c-32bd-4e58-978d-c85649cb8583 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:56.998602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640ea4c-32bd-4e58-978d-c85649cb8583 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:56.998923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640ea4c-32bd-4e58-978d-c85649cb8583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:57.066563 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1640ea4c-32bd-4e58-978d-c85649cb8583 None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:57.067794 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1640ea4c-32bd-4e58-978d-c85649cb8583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:57.154740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 707/1397] 10.222.0.58 () {58 vars in 1009 bytes} [Wed Aug 24 12:25:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:57.197717 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c7567-ffa9-4b6e-8a9f-35192a0fabfd None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:57.198204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c7567-ffa9-4b6e-8a9f-35192a0fabfd None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:57.198310 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10c7567-ffa9-4b6e-8a9f-35192a0fabfd None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:57.200516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 691/1398] 10.222.0.58 () {54 vars in 943 bytes} [Wed Aug 24 12:25:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:57.206856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11d218-0dbd-4eb7-b48e-96413d8b0df5 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:57.207215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11d218-0dbd-4eb7-b48e-96413d8b0df5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:57.207493 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11d218-0dbd-4eb7-b48e-96413d8b0df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:57.267569 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:57.286706 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9d11d218-0dbd-4eb7-b48e-96413d8b0df5 None None] MFA Rules not processed for user `67756e31aeee48a897cfa8bd35a10119`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:57.287998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9d11d218-0dbd-4eb7-b48e-96413d8b0df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:57.403304 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 708/1399] 10.222.0.58 () {58 vars in 1040 bytes} [Wed Aug 24 12:25:57 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:57.412535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c7567-ffa9-4b6e-8a9f-35192a0fabfd None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:57.414088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f10c7567-ffa9-4b6e-8a9f-35192a0fabfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:57.492512 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:57.558174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10c7567-ffa9-4b6e-8a9f-35192a0fabfd None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:57.665614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e312c-bb0f-48bf-ad2f-03518dddf964 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:57.666143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e312c-bb0f-48bf-ad2f-03518dddf964 service swift] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:57.666421 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e312c-bb0f-48bf-ad2f-03518dddf964 service swift] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:57.666697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e312c-bb0f-48bf-ad2f-03518dddf964 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:57.668542 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-612e312c-bb0f-48bf-ad2f-03518dddf964 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:57.728221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:57.754415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e312c-bb0f-48bf-ad2f-03518dddf964 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:57.756081 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e312c-bb0f-48bf-ad2f-03518dddf964 service swift] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:57.765093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 692/1400] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:25:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:25:57.808376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901a140-c65d-4e89-9a96-a6397a96aa2d None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:57.809318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901a140-c65d-4e89-9a96-a6397a96aa2d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:57.809663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901a140-c65d-4e89-9a96-a6397a96aa2d None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:57.811100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 709/1401] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:25:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:25:57.816653 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972387a6-0a82-4145-a947-419e48e7d9d5 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:57.816949 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972387a6-0a82-4145-a947-419e48e7d9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:57.817229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972387a6-0a82-4145-a947-419e48e7d9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:57.900398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-972387a6-0a82-4145-a947-419e48e7d9d5 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:25:57.901720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-972387a6-0a82-4145-a947-419e48e7d9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:58.007767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 693/1402] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:25:58.318052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901a140-c65d-4e89-9a96-a6397a96aa2d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:25:58.319634 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7901a140-c65d-4e89-9a96-a6397a96aa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:58.446031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:58.492956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901a140-c65d-4e89-9a96-a6397a96aa2d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:25:58.620363 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66341765-9d09-4fa0-92dc-e0dfcc3b22ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:25:58.621629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66341765-9d09-4fa0-92dc-e0dfcc3b22ed service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:25:58.621910 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66341765-9d09-4fa0-92dc-e0dfcc3b22ed service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:25:58.622185 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66341765-9d09-4fa0-92dc-e0dfcc3b22ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:25:58.623842 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-66341765-9d09-4fa0-92dc-e0dfcc3b22ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:25:58.707765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:25:58.731277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66341765-9d09-4fa0-92dc-e0dfcc3b22ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:25:58.732925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66341765-9d09-4fa0-92dc-e0dfcc3b22ed service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:25:58.754004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 710/1403] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:25:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:00.713709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80d45f-c40c-46b8-938c-3fbbf3934457 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:00.714300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80d45f-c40c-46b8-938c-3fbbf3934457 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:00.714300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80d45f-c40c-46b8-938c-3fbbf3934457 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:00.716196 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 694/1404] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:26:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 12:26:00.722077 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779f6c8-01fb-40b4-b004-81da560272a1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:00.722485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779f6c8-01fb-40b4-b004-81da560272a1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:00.723295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7779f6c8-01fb-40b4-b004-81da560272a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:00.796587 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7779f6c8-01fb-40b4-b004-81da560272a1 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:00.798278 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7779f6c8-01fb-40b4-b004-81da560272a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:00.913396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 711/1405] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:01.214812 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80d45f-c40c-46b8-938c-3fbbf3934457 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:01.216663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-de80d45f-c40c-46b8-938c-3fbbf3934457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:01.293499 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:01.404014 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80d45f-c40c-46b8-938c-3fbbf3934457 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:01.536807 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f43eef-c8f9-4978-9642-6523cdb1d522 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:01.537503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f43eef-c8f9-4978-9642-6523cdb1d522 service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:01.537789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f43eef-c8f9-4978-9642-6523cdb1d522 service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:01.538067 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f43eef-c8f9-4978-9642-6523cdb1d522 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:01.540061 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-64f43eef-c8f9-4978-9642-6523cdb1d522 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:01.552602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:01.638461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f43eef-c8f9-4978-9642-6523cdb1d522 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:01.640101 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f43eef-c8f9-4978-9642-6523cdb1d522 service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:01.658901 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 695/1406] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:03.672054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a81f8-b215-4b5e-8319-a11ce2a4e5b8 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:03.672583 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a81f8-b215-4b5e-8319-a11ce2a4e5b8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:03.672583 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a81f8-b215-4b5e-8319-a11ce2a4e5b8 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:03.674130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 712/1407] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:03.681435 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dce4ca-ee60-4580-83fd-7b4f097a57c2 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:03.681435 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dce4ca-ee60-4580-83fd-7b4f097a57c2 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:03.681435 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dce4ca-ee60-4580-83fd-7b4f097a57c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:03.751108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-d2dce4ca-ee60-4580-83fd-7b4f097a57c2 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:03.752886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d2dce4ca-ee60-4580-83fd-7b4f097a57c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:03.854708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:03.870352 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 696/1408] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:04.163211 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a81f8-b215-4b5e-8319-a11ce2a4e5b8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:04.164832 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-967a81f8-b215-4b5e-8319-a11ce2a4e5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:04.229634 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:04.331905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a81f8-b215-4b5e-8319-a11ce2a4e5b8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:04.469641 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0085c0d-9c68-4b8e-9516-6121236bc06b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:04.471782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0085c0d-9c68-4b8e-9516-6121236bc06b service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:04.471782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0085c0d-9c68-4b8e-9516-6121236bc06b service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:04.471782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0085c0d-9c68-4b8e-9516-6121236bc06b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:04.472406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e0085c0d-9c68-4b8e-9516-6121236bc06b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:04.482423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:04.576925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0085c0d-9c68-4b8e-9516-6121236bc06b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:04.578621 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0085c0d-9c68-4b8e-9516-6121236bc06b service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:04.598748 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 713/1409] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:13.868428 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=5 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:14.600078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=41 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:20.369118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d861fd-8552-46c8-90bd-d0bdff759bed None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:20.369735 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d861fd-8552-46c8-90bd-d0bdff759bed None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:20.369735 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d861fd-8552-46c8-90bd-d0bdff759bed None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:20.371673 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 697/1410] 10.222.0.58 () {54 vars in 918 bytes} [Wed Aug 24 12:26:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:20.376926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8ee0c-5a70-49a0-9fcf-466ea5523ea1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:20.377361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8ee0c-5a70-49a0-9fcf-466ea5523ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:20.377464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8ee0c-5a70-49a0-9fcf-466ea5523ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:20.452326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9eb8ee0c-5a70-49a0-9fcf-466ea5523ea1 None None] MFA Rules not processed for user `b7ba45e3cff74962965b248f9f017f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:20.453535 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9eb8ee0c-5a70-49a0-9fcf-466ea5523ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:20.559089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 714/1411] 10.222.0.58 () {58 vars in 1015 bytes} [Wed Aug 24 12:26:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:20.574007 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc233f8-fc06-4578-80a4-27ab72503cb5 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:20.574295 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc233f8-fc06-4578-80a4-27ab72503cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:20.574567 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc233f8-fc06-4578-80a4-27ab72503cb5 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:20.576267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 698/1412] 10.222.0.58 () {54 vars in 918 bytes} [Wed Aug 24 12:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:20.580907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5e8c5-2678-4588-b85f-717e9bb78fb0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:20.581190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5e8c5-2678-4588-b85f-717e9bb78fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:20.581458 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5e8c5-2678-4588-b85f-717e9bb78fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:20.657836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:20.660988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0ff5e8c5-2678-4588-b85f-717e9bb78fb0 None None] MFA Rules not processed for user `311dbed3ab0b44a7830a93227ff9ad33`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:20.662705 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0ff5e8c5-2678-4588-b85f-717e9bb78fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:20.781611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 715/1413] 10.222.0.58 () {58 vars in 1015 bytes} [Wed Aug 24 12:26:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:26:23.163591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4796b46-5fec-4c65-a149-1d4d4df5f344 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:23.164162 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4796b46-5fec-4c65-a149-1d4d4df5f344 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:23.164162 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4796b46-5fec-4c65-a149-1d4d4df5f344 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:23.165472 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:23.171324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915516e-0012-4a3d-bdcb-a9828e05ca3d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:23.171808 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915516e-0012-4a3d-bdcb-a9828e05ca3d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:23.171921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915516e-0012-4a3d-bdcb-a9828e05ca3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:23.237713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5915516e-0012-4a3d-bdcb-a9828e05ca3d None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:23.238905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5915516e-0012-4a3d-bdcb-a9828e05ca3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:23.293294 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:23.335759 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:23.624656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4796b46-5fec-4c65-a149-1d4d4df5f344 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:23.626221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d4796b46-5fec-4c65-a149-1d4d4df5f344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:23.772363 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4796b46-5fec-4c65-a149-1d4d4df5f344 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:23.843817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:23.885540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a2c11-276b-4fc3-b37b-3198404ab897 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:23.886267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a2c11-276b-4fc3-b37b-3198404ab897 service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:23.886574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a2c11-276b-4fc3-b37b-3198404ab897 service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:23.886832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a2c11-276b-4fc3-b37b-3198404ab897 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:23.888484 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fb8a2c11-276b-4fc3-b37b-3198404ab897 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:23.976475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a2c11-276b-4fc3-b37b-3198404ab897 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:23.978076 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a2c11-276b-4fc3-b37b-3198404ab897 service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:23.996897 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 700/1416] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:24.906529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1c52b-ed2f-4ef8-bc4b-91c2cc517c3b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:24.906981 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1c52b-ed2f-4ef8-bc4b-91c2cc517c3b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:24.907095 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1c52b-ed2f-4ef8-bc4b-91c2cc517c3b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:24.908459 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:24.913396 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e53d4-4478-449a-b36b-4332b26f5a66 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:24.913809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e53d4-4478-449a-b36b-4332b26f5a66 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:24.913947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e53d4-4478-449a-b36b-4332b26f5a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:24.988494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-944e53d4-4478-449a-b36b-4332b26f5a66 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:24.989700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-944e53d4-4478-449a-b36b-4332b26f5a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:25.017888 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:25.088354 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:25.242759 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1c52b-ed2f-4ef8-bc4b-91c2cc517c3b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:25.244200 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-69b1c52b-ed2f-4ef8-bc4b-91c2cc517c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:25.392745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1c52b-ed2f-4ef8-bc4b-91c2cc517c3b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:25.431762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:25.505300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9145d-a5ea-4f49-b6ed-58faa6805660 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:25.505831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9145d-a5ea-4f49-b6ed-58faa6805660 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:25.506112 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9145d-a5ea-4f49-b6ed-58faa6805660 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:25.506386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9145d-a5ea-4f49-b6ed-58faa6805660 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:25.508014 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-99e9145d-a5ea-4f49-b6ed-58faa6805660 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:25.597617 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9145d-a5ea-4f49-b6ed-58faa6805660 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:25.599206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9145d-a5ea-4f49-b6ed-58faa6805660 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:25.617449 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 718/1419] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:26.498096 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271a361-5048-4e2f-996a-34968e5a32e2 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:26.498708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271a361-5048-4e2f-996a-34968e5a32e2 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:26.498708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271a361-5048-4e2f-996a-34968e5a32e2 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:26.500048 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:26.504910 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514015c5-4bb0-465d-a8b6-310a52356de8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:26.505360 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514015c5-4bb0-465d-a8b6-310a52356de8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:26.505458 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514015c5-4bb0-465d-a8b6-310a52356de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:26.561492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:26.584667 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-514015c5-4bb0-465d-a8b6-310a52356de8 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:26.585882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-514015c5-4bb0-465d-a8b6-310a52356de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:26.682118 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:26.853425 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271a361-5048-4e2f-996a-34968e5a32e2 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:26.855771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1271a361-5048-4e2f-996a-34968e5a32e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:27.004654 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271a361-5048-4e2f-996a-34968e5a32e2 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:27.017210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:27.123527 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed1b46-f998-4324-ab25-d54ac6a9ee45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:27.124077 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed1b46-f998-4324-ab25-d54ac6a9ee45 service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:27.124352 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed1b46-f998-4324-ab25-d54ac6a9ee45 service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:27.124625 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed1b46-f998-4324-ab25-d54ac6a9ee45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:27.126227 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-04ed1b46-f998-4324-ab25-d54ac6a9ee45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:27.216737 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed1b46-f998-4324-ab25-d54ac6a9ee45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:27.218346 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed1b46-f998-4324-ab25-d54ac6a9ee45 service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:27.236510 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 703/1422] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:28.096190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99852d98-2c73-4259-b22c-55b0a7b79d00 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:28.096757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99852d98-2c73-4259-b22c-55b0a7b79d00 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:28.096757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99852d98-2c73-4259-b22c-55b0a7b79d00 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:28.098159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:28.103193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c4944-70ad-4e77-bee0-eacacc6c5144 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:28.103592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c4944-70ad-4e77-bee0-eacacc6c5144 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:28.103807 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c4944-70ad-4e77-bee0-eacacc6c5144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:28.124855 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:28.180440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-377c4944-70ad-4e77-bee0-eacacc6c5144 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:28.181563 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-377c4944-70ad-4e77-bee0-eacacc6c5144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:28.275616 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:28.436081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99852d98-2c73-4259-b22c-55b0a7b79d00 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:28.437556 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-99852d98-2c73-4259-b22c-55b0a7b79d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:28.546522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:28.583004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99852d98-2c73-4259-b22c-55b0a7b79d00 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:28.694348 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42fa94-6b56-49c5-8921-4c9c3bcca64c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:28.694878 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42fa94-6b56-49c5-8921-4c9c3bcca64c service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:28.695159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42fa94-6b56-49c5-8921-4c9c3bcca64c service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:28.695432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42fa94-6b56-49c5-8921-4c9c3bcca64c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:28.697330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1b42fa94-6b56-49c5-8921-4c9c3bcca64c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:28.776266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:28.790999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42fa94-6b56-49c5-8921-4c9c3bcca64c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:28.793254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42fa94-6b56-49c5-8921-4c9c3bcca64c service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:28.811896 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 721/1425] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:29.670674 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ba559-ab4c-44cf-ad28-ab43c90062ff None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:29.671204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ba559-ab4c-44cf-ad28-ab43c90062ff None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:29.671204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ba559-ab4c-44cf-ad28-ab43c90062ff None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:29.672635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:29.677559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b2ff2-2973-4f9a-92c6-43c4cfdd85e2 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:29.677944 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b2ff2-2973-4f9a-92c6-43c4cfdd85e2 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:29.678119 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b2ff2-2973-4f9a-92c6-43c4cfdd85e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:29.755319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ab9b2ff2-2973-4f9a-92c6-43c4cfdd85e2 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:29.756524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ab9b2ff2-2973-4f9a-92c6-43c4cfdd85e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:29.851603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:30.002168 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ba559-ab4c-44cf-ad28-ab43c90062ff None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:30.003736 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-cb0ba559-ab4c-44cf-ad28-ab43c90062ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:30.086699 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:30.149849 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ba559-ab4c-44cf-ad28-ab43c90062ff None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:30.263744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ad108-0071-4b83-8697-6d008b7af77d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:30.264385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ad108-0071-4b83-8697-6d008b7af77d service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:30.264667 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ad108-0071-4b83-8697-6d008b7af77d service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:30.265066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ad108-0071-4b83-8697-6d008b7af77d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:30.266792 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2e2ad108-0071-4b83-8697-6d008b7af77d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:30.318102 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:30.359438 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2ad108-0071-4b83-8697-6d008b7af77d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:30.361084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2ad108-0071-4b83-8697-6d008b7af77d service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:30.379498 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 706/1428] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:31.185959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655c64b-bf82-4933-be98-01bdef29288d None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:31.186504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655c64b-bf82-4933-be98-01bdef29288d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:31.186504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655c64b-bf82-4933-be98-01bdef29288d None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:31.187865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:31.192852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae3d4f-452e-44ed-b84d-df2eb22f5fe1 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:31.193284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae3d4f-452e-44ed-b84d-df2eb22f5fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:31.193477 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae3d4f-452e-44ed-b84d-df2eb22f5fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:31.269559 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-24ae3d4f-452e-44ed-b84d-df2eb22f5fe1 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:31.270766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-24ae3d4f-452e-44ed-b84d-df2eb22f5fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:31.366064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:31.517607 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655c64b-bf82-4933-be98-01bdef29288d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:31.519102 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9655c64b-bf82-4933-be98-01bdef29288d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:31.568987 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:31.666376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655c64b-bf82-4933-be98-01bdef29288d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:31.777971 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7172eab-0a1c-46c5-93c8-0425544ff6a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:31.778477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7172eab-0a1c-46c5-93c8-0425544ff6a2 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:31.778759 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7172eab-0a1c-46c5-93c8-0425544ff6a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:31.779035 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7172eab-0a1c-46c5-93c8-0425544ff6a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:31.780673 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f7172eab-0a1c-46c5-93c8-0425544ff6a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:31.787495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:31.870817 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7172eab-0a1c-46c5-93c8-0425544ff6a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:31.872481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7172eab-0a1c-46c5-93c8-0425544ff6a2 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:31.891289 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 724/1431] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:32.696869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8d8b1-fe9f-4495-bdab-94635a4eb44a None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:32.697429 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8d8b1-fe9f-4495-bdab-94635a4eb44a None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:32.697429 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8d8b1-fe9f-4495-bdab-94635a4eb44a None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:32.699421 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:26:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:32.704640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e620c-f012-4d70-9bbe-3e0cdff7593e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:32.705072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e620c-f012-4d70-9bbe-3e0cdff7593e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:32.705169 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e620c-f012-4d70-9bbe-3e0cdff7593e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:32.797934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c71e620c-f012-4d70-9bbe-3e0cdff7593e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:32.799071 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c71e620c-f012-4d70-9bbe-3e0cdff7593e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:32.867722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:32.897707 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:33.048739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8d8b1-fe9f-4495-bdab-94635a4eb44a None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:33.050245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-8bd8d8b1-fe9f-4495-bdab-94635a4eb44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:33.068516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:33.203986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8d8b1-fe9f-4495-bdab-94635a4eb44a None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:33.290868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:33.316986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce85eb-d18d-4a44-afbc-9f77ad2b1e88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:33.317594 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce85eb-d18d-4a44-afbc-9f77ad2b1e88 service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:33.317881 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce85eb-d18d-4a44-afbc-9f77ad2b1e88 service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:33.318154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce85eb-d18d-4a44-afbc-9f77ad2b1e88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:33.319809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b1ce85eb-d18d-4a44-afbc-9f77ad2b1e88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:33.409106 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce85eb-d18d-4a44-afbc-9f77ad2b1e88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:33.410731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce85eb-d18d-4a44-afbc-9f77ad2b1e88 service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:33.428645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 709/1434] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:34.249717 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919c6c1-9f41-4122-8f4e-eb6cf36634b0 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:34.250355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919c6c1-9f41-4122-8f4e-eb6cf36634b0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:34.250355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919c6c1-9f41-4122-8f4e-eb6cf36634b0 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:34.251607 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:34.256552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b179b-1863-4f53-9520-6e524cce6b16 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:34.257008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b179b-1863-4f53-9520-6e524cce6b16 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:34.257138 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b179b-1863-4f53-9520-6e524cce6b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:34.332960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-a89b179b-1863-4f53-9520-6e524cce6b16 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:34.334173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a89b179b-1863-4f53-9520-6e524cce6b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:34.375836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:34.437172 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:26:34.589004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919c6c1-9f41-4122-8f4e-eb6cf36634b0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:34.591447 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e919c6c1-9f41-4122-8f4e-eb6cf36634b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:34.735966 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e919c6c1-9f41-4122-8f4e-eb6cf36634b0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:34.800213 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:34.860629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aac3d7-f252-4671-a908-1b44ab1c847b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:34.861239 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aac3d7-f252-4671-a908-1b44ab1c847b service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:34.861522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aac3d7-f252-4671-a908-1b44ab1c847b service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:34.861795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aac3d7-f252-4671-a908-1b44ab1c847b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:34.863398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f6aac3d7-f252-4671-a908-1b44ab1c847b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:34.953518 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aac3d7-f252-4671-a908-1b44ab1c847b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:34.955120 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aac3d7-f252-4671-a908-1b44ab1c847b service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:34.972908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 727/1437] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:35.906514 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4b7d5-8f20-4790-889b-e1ad5934396d None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:35.907076 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4b7d5-8f20-4790-889b-e1ad5934396d None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:35.907076 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4b7d5-8f20-4790-889b-e1ad5934396d None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:35.908502 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:35.914581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae652982-59d1-42db-ade1-b0246937fc6b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:35.915072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae652982-59d1-42db-ade1-b0246937fc6b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:35.915230 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae652982-59d1-42db-ade1-b0246937fc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:35.999213 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ae652982-59d1-42db-ade1-b0246937fc6b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:36.000544 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ae652982-59d1-42db-ade1-b0246937fc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:36.002747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:36.122373 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:36.287873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4b7d5-8f20-4790-889b-e1ad5934396d None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:36.289346 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e7e4b7d5-8f20-4790-889b-e1ad5934396d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:36.437799 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4b7d5-8f20-4790-889b-e1ad5934396d None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:36.463649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:36.555904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3308c35-c74a-4152-9742-6986aa55e08d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:36.556651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3308c35-c74a-4152-9742-6986aa55e08d service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:36.556931 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3308c35-c74a-4152-9742-6986aa55e08d service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:36.557202 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3308c35-c74a-4152-9742-6986aa55e08d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:36.559370 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a3308c35-c74a-4152-9742-6986aa55e08d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:36.648756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3308c35-c74a-4152-9742-6986aa55e08d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:36.650377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3308c35-c74a-4152-9742-6986aa55e08d service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:36.668476 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 712/1440] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:38.489808 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f2440-780b-4a98-87c4-f083832cc531 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:38.490418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f2440-780b-4a98-87c4-f083832cc531 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:38.490418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f2440-780b-4a98-87c4-f083832cc531 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:38.491734 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:38.497151 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7dfb8-844f-430e-8cdd-e3458e9be317 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:38.497566 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7dfb8-844f-430e-8cdd-e3458e9be317 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:38.497739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7dfb8-844f-430e-8cdd-e3458e9be317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:38.539054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:38.564419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-2eb7dfb8-844f-430e-8cdd-e3458e9be317 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:38.565639 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2eb7dfb8-844f-430e-8cdd-e3458e9be317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:38.661927 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:38.950546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f2440-780b-4a98-87c4-f083832cc531 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:38.952124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-609f2440-780b-4a98-87c4-f083832cc531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:39.054294 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f2440-780b-4a98-87c4-f083832cc531 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:39.086464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:39.127711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311a1cb-43dc-48fb-9337-b7654a7e7c8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:39.128410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311a1cb-43dc-48fb-9337-b7654a7e7c8e service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:39.128695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311a1cb-43dc-48fb-9337-b7654a7e7c8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:39.128969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311a1cb-43dc-48fb-9337-b7654a7e7c8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:39.130601 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0311a1cb-43dc-48fb-9337-b7654a7e7c8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:39.243057 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311a1cb-43dc-48fb-9337-b7654a7e7c8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:39.244530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311a1cb-43dc-48fb-9337-b7654a7e7c8e service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:39.262884 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 730/1443] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:41.368358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8dc10-dd0c-4f09-8a69-1941f3cc2161 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:41.368964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8dc10-dd0c-4f09-8a69-1941f3cc2161 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:41.368964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8dc10-dd0c-4f09-8a69-1941f3cc2161 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:41.370321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:41.375892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31e9f3-1d16-4fd4-b823-2878dd319ca0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:41.376330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31e9f3-1d16-4fd4-b823-2878dd319ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:41.376438 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31e9f3-1d16-4fd4-b823-2878dd319ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:41.442856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4b31e9f3-1d16-4fd4-b823-2878dd319ca0 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:41.444073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4b31e9f3-1d16-4fd4-b823-2878dd319ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:41.457601 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:41.543405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:41.828349 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8dc10-dd0c-4f09-8a69-1941f3cc2161 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:41.829961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c0e8dc10-dd0c-4f09-8a69-1941f3cc2161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:41.933450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8dc10-dd0c-4f09-8a69-1941f3cc2161 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:41.945614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:42.006582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65297f9c-e988-43d7-a3c7-b7480c6a9b2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:42.007187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65297f9c-e988-43d7-a3c7-b7480c6a9b2b service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:42.007465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65297f9c-e988-43d7-a3c7-b7480c6a9b2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:42.007770 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65297f9c-e988-43d7-a3c7-b7480c6a9b2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:42.009369 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-65297f9c-e988-43d7-a3c7-b7480c6a9b2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:42.118822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65297f9c-e988-43d7-a3c7-b7480c6a9b2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:42.120235 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65297f9c-e988-43d7-a3c7-b7480c6a9b2b service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:42.139269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 715/1446] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:44.347473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8103c-4e72-4b50-be8a-5c73f45c108b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:44.348074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8103c-4e72-4b50-be8a-5c73f45c108b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:44.348074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8103c-4e72-4b50-be8a-5c73f45c108b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:44.349390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:44.354576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b4954-5c20-4f6c-ac82-f7379564bbd7 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:44.354966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b4954-5c20-4f6c-ac82-f7379564bbd7 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:44.355374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b4954-5c20-4f6c-ac82-f7379564bbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:44.404446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:44.422093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-c13b4954-5c20-4f6c-ac82-f7379564bbd7 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:44.423305 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c13b4954-5c20-4f6c-ac82-f7379564bbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:44.518510 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:44.800420 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8103c-4e72-4b50-be8a-5c73f45c108b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:44.801948 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-56c8103c-4e72-4b50-be8a-5c73f45c108b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:44.908822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8103c-4e72-4b50-be8a-5c73f45c108b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:44.949233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:44.982329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5fe11d-72f5-43f8-88e5-90db3a7bc8d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:44.982923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fe11d-72f5-43f8-88e5-90db3a7bc8d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:44.983201 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fe11d-72f5-43f8-88e5-90db3a7bc8d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:44.983469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fe11d-72f5-43f8-88e5-90db3a7bc8d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:44.985087 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3d5fe11d-72f5-43f8-88e5-90db3a7bc8d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:45.091640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5fe11d-72f5-43f8-88e5-90db3a7bc8d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:45.093077 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5fe11d-72f5-43f8-88e5-90db3a7bc8d1 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:45.111309 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 733/1449] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:47.156258 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6a5f5-7e11-4c1f-9eb4-3930e546edeb None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:47.156787 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6a5f5-7e11-4c1f-9eb4-3930e546edeb None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:47.156787 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e6a5f5-7e11-4c1f-9eb4-3930e546edeb None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:47.158122 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:47.163674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50a21e-4613-4739-b3f6-6d968dc27a3f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:47.164074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50a21e-4613-4739-b3f6-6d968dc27a3f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:47.164230 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50a21e-4613-4739-b3f6-6d968dc27a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:47.233695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7c50a21e-4613-4739-b3f6-6d968dc27a3f None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:47.234925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7c50a21e-4613-4739-b3f6-6d968dc27a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:47.255548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:47.336038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:47.622964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6a5f5-7e11-4c1f-9eb4-3930e546edeb None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:47.624541 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c5e6a5f5-7e11-4c1f-9eb4-3930e546edeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:47.730745 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e6a5f5-7e11-4c1f-9eb4-3930e546edeb None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:47.750205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:47.803716 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fe302-9b36-4d79-b8c0-4bdf30468143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:47.804320 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fe302-9b36-4d79-b8c0-4bdf30468143 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:47.804596 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fe302-9b36-4d79-b8c0-4bdf30468143 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:47.804865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fe302-9b36-4d79-b8c0-4bdf30468143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:47.806431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7c9fe302-9b36-4d79-b8c0-4bdf30468143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:47.914857 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fe302-9b36-4d79-b8c0-4bdf30468143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:47.916324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fe302-9b36-4d79-b8c0-4bdf30468143 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:47.934578 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 718/1452] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:49.899636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf780364-1039-421f-83ae-e3658fae407c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:49.900142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf780364-1039-421f-83ae-e3658fae407c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:49.900259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf780364-1039-421f-83ae-e3658fae407c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:49.901607 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:49.907549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7f92d-083c-46d7-89b4-decd3498120a None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:49.908000 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7f92d-083c-46d7-89b4-decd3498120a None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:49.908159 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7f92d-083c-46d7-89b4-decd3498120a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:50.091081 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:50.093701 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-34d7f92d-083c-46d7-89b4-decd3498120a None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:50.095414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-34d7f92d-083c-46d7-89b4-decd3498120a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:50.190750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:50.487927 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf780364-1039-421f-83ae-e3658fae407c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:50.489495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bf780364-1039-421f-83ae-e3658fae407c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:50.594051 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf780364-1039-421f-83ae-e3658fae407c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:50.642841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:50.667986 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9dad4-1768-44f8-b03e-b2fcf39ec3a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:50.668585 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9dad4-1768-44f8-b03e-b2fcf39ec3a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:50.668863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9dad4-1768-44f8-b03e-b2fcf39ec3a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:50.669134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9dad4-1768-44f8-b03e-b2fcf39ec3a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:50.670723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b8c9dad4-1768-44f8-b03e-b2fcf39ec3a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:50.779841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9dad4-1768-44f8-b03e-b2fcf39ec3a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:50.781206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9dad4-1768-44f8-b03e-b2fcf39ec3a2 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:50.800663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 736/1455] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:52.788674 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0bc1c-b82a-4dd5-9826-f1a274858dde None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:52.789245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0bc1c-b82a-4dd5-9826-f1a274858dde None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:52.789245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0bc1c-b82a-4dd5-9826-f1a274858dde None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:52.790554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:52.796247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c217f-8e48-4a6d-a6b5-cee1e74a3887 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:52.796664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c217f-8e48-4a6d-a6b5-cee1e74a3887 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:52.796809 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c217f-8e48-4a6d-a6b5-cee1e74a3887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:52.864258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-488c217f-8e48-4a6d-a6b5-cee1e74a3887 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:52.865472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-488c217f-8e48-4a6d-a6b5-cee1e74a3887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:52.897581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:52.970567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:26:53.254485 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0bc1c-b82a-4dd5-9826-f1a274858dde None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:53.256155 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-44e0bc1c-b82a-4dd5-9826-f1a274858dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:53.361943 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0bc1c-b82a-4dd5-9826-f1a274858dde None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:53.390211 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:53.435790 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b58eb1-9f33-4148-9390-f823f0cf1050 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:53.436397 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b58eb1-9f33-4148-9390-f823f0cf1050 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:53.436677 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b58eb1-9f33-4148-9390-f823f0cf1050 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:53.436947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b58eb1-9f33-4148-9390-f823f0cf1050 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:53.438539 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e1b58eb1-9f33-4148-9390-f823f0cf1050 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:53.547775 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b58eb1-9f33-4148-9390-f823f0cf1050 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:53.549117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b58eb1-9f33-4148-9390-f823f0cf1050 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:53.566661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 721/1458] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:55.547107 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7bd60-cfb9-4ac7-b164-56aa6defb85b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:55.547733 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7bd60-cfb9-4ac7-b164-56aa6defb85b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:55.547733 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7bd60-cfb9-4ac7-b164-56aa6defb85b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:55.549218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:55.554655 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571ebe6-8b71-42f8-8ab9-8b91696328d3 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:55.555071 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571ebe6-8b71-42f8-8ab9-8b91696328d3 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:55.555199 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571ebe6-8b71-42f8-8ab9-8b91696328d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:55.625388 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-0571ebe6-8b71-42f8-8ab9-8b91696328d3 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:55.626595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-0571ebe6-8b71-42f8-8ab9-8b91696328d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:55.634756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:55.723407 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:56.011914 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7bd60-cfb9-4ac7-b164-56aa6defb85b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:56.013954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-95a7bd60-cfb9-4ac7-b164-56aa6defb85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:56.119423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7bd60-cfb9-4ac7-b164-56aa6defb85b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:56.175967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:56.194388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f7930-585e-4757-bb2c-dc84aee190f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:56.195039 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f7930-585e-4757-bb2c-dc84aee190f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:56.195522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f7930-585e-4757-bb2c-dc84aee190f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:56.196059 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f7930-585e-4757-bb2c-dc84aee190f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:56.198307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4e0f7930-585e-4757-bb2c-dc84aee190f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:56.315277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f7930-585e-4757-bb2c-dc84aee190f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:56.316738 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f7930-585e-4757-bb2c-dc84aee190f7 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:56.335130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 739/1461] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:26:58.315271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6575c-93ca-4509-9f87-6e78be3abb6e None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:58.315821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6575c-93ca-4509-9f87-6e78be3abb6e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:58.315912 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e6575c-93ca-4509-9f87-6e78be3abb6e None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:58.317213 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:26:58.322724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528037f-e2b0-43f4-b950-73a7135ab891 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:58.323126 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528037f-e2b0-43f4-b950-73a7135ab891 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:58.323298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528037f-e2b0-43f4-b950-73a7135ab891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:58.388938 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f528037f-e2b0-43f4-b950-73a7135ab891 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:26:58.390146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f528037f-e2b0-43f4-b950-73a7135ab891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:58.428591 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:58.486868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:26:58.770346 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6575c-93ca-4509-9f87-6e78be3abb6e None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:26:58.771896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-76e6575c-93ca-4509-9f87-6e78be3abb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:58.873070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e6575c-93ca-4509-9f87-6e78be3abb6e None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:26:58.911246 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:26:58.946862 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320e30f-8520-43d8-aee6-d3b90c3dba09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:26:58.947751 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320e30f-8520-43d8-aee6-d3b90c3dba09 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:26:58.948030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320e30f-8520-43d8-aee6-d3b90c3dba09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:26:58.948307 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320e30f-8520-43d8-aee6-d3b90c3dba09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:26:58.952908 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7320e30f-8520-43d8-aee6-d3b90c3dba09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:26:59.063336 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320e30f-8520-43d8-aee6-d3b90c3dba09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:26:59.064818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320e30f-8520-43d8-aee6-d3b90c3dba09 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:26:59.083359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 724/1464] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:26:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:00.990626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caca3e0-6305-43f7-b43d-3aa5b03c59aa None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:00.991188 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caca3e0-6305-43f7-b43d-3aa5b03c59aa None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:00.991188 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caca3e0-6305-43f7-b43d-3aa5b03c59aa None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:00.992563 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:00.997836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc803a04-566d-4588-9317-96db78603fbd None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:00.998266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc803a04-566d-4588-9317-96db78603fbd None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:00.998414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc803a04-566d-4588-9317-96db78603fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:01.065938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-bc803a04-566d-4588-9317-96db78603fbd None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:01.067162 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-bc803a04-566d-4588-9317-96db78603fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:01.085526 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:01.164012 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:01.448214 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caca3e0-6305-43f7-b43d-3aa5b03c59aa None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:01.450496 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9caca3e0-6305-43f7-b43d-3aa5b03c59aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:01.557434 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caca3e0-6305-43f7-b43d-3aa5b03c59aa None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:01.626520 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:01.633793 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d046d-2016-4a0b-9a9f-4f2deb8b2283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:01.634398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d046d-2016-4a0b-9a9f-4f2deb8b2283 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:01.634678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d046d-2016-4a0b-9a9f-4f2deb8b2283 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:01.634957 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d046d-2016-4a0b-9a9f-4f2deb8b2283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:01.636675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-768d046d-2016-4a0b-9a9f-4f2deb8b2283 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:01.749641 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d046d-2016-4a0b-9a9f-4f2deb8b2283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:01.750998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d046d-2016-4a0b-9a9f-4f2deb8b2283 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:01.769245 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 742/1467] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:03.716829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bf68a-a63f-47b6-8575-3b6253c989e4 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:03.717386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bf68a-a63f-47b6-8575-3b6253c989e4 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:03.717386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bf68a-a63f-47b6-8575-3b6253c989e4 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:03.718737 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:03.724185 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331df21-7866-4902-ae85-3677ea0c4026 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:03.724622 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331df21-7866-4902-ae85-3677ea0c4026 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:03.724737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331df21-7866-4902-ae85-3677ea0c4026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:03.790483 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6331df21-7866-4902-ae85-3677ea0c4026 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:03.791723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6331df21-7866-4902-ae85-3677ea0c4026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:03.840302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:03.890081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:04.179379 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bf68a-a63f-47b6-8575-3b6253c989e4 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:04.181033 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f90bf68a-a63f-47b6-8575-3b6253c989e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:04.283393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bf68a-a63f-47b6-8575-3b6253c989e4 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:04.334696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:04.360093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba7f72-891e-431e-b8b2-20725de8d973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:04.360687 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba7f72-891e-431e-b8b2-20725de8d973 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:04.360963 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba7f72-891e-431e-b8b2-20725de8d973 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:04.361231 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba7f72-891e-431e-b8b2-20725de8d973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:04.362801 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a2ba7f72-891e-431e-b8b2-20725de8d973 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:04.471378 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba7f72-891e-431e-b8b2-20725de8d973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:04.472774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba7f72-891e-431e-b8b2-20725de8d973 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:04.490466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 727/1470] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:13.888206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=22 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:14.488220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=66 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:24.086102 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1015a-ca8c-409c-9ad1-d3cfae194af6 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:24.086716 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1015a-ca8c-409c-9ad1-d3cfae194af6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:24.086716 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1015a-ca8c-409c-9ad1-d3cfae194af6 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:24.088052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 744/1471] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:24.093244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7f265-fb3f-4ecf-8731-532af816e835 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:24.093641 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7f265-fb3f-4ecf-8731-532af816e835 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:24.093807 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7f265-fb3f-4ecf-8731-532af816e835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:24.159347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-2fe7f265-fb3f-4ecf-8731-532af816e835 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:24.160492 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2fe7f265-fb3f-4ecf-8731-532af816e835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:24.256901 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 728/1472] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:24.564501 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a07621-7c58-4be5-8b25-39635b535a6b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:24.564788 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a07621-7c58-4be5-8b25-39635b535a6b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:24.565067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a07621-7c58-4be5-8b25-39635b535a6b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:24.566471 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 745/1473] 10.222.0.58 () {54 vars in 967 bytes} [Wed Aug 24 12:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:24.571308 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543f970-b572-4c8f-ad7f-bee11a216fa7 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:24.571595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543f970-b572-4c8f-ad7f-bee11a216fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:24.571903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543f970-b572-4c8f-ad7f-bee11a216fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:24.645863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-e543f970-b572-4c8f-ad7f-bee11a216fa7 None None] MFA Rules not processed for user `dd8df2f3357d4660bc22366f9746f89c`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:24.647075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e543f970-b572-4c8f-ad7f-bee11a216fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:24.668241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:24.722959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 729/1474] 10.222.0.58 () {58 vars in 1064 bytes} [Wed Aug 24 12:27:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:24.731196 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a07621-7c58-4be5-8b25-39635b535a6b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:24.732762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d8a07621-7c58-4be5-8b25-39635b535a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:24.835377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a07621-7c58-4be5-8b25-39635b535a6b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:24.908406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e25f7-7c64-4cdf-bf6f-5fdb218af768 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd8df2f3357d4660bc22366f9746f89c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:24.909009 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e25f7-7c64-4cdf-bf6f-5fdb218af768 service glance] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:24.909287 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e25f7-7c64-4cdf-bf6f-5fdb218af768 service glance] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:24.909558 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e25f7-7c64-4cdf-bf6f-5fdb218af768 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:24.911132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-706e25f7-7c64-4cdf-bf6f-5fdb218af768 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:24.965246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:25.021177 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e25f7-7c64-4cdf-bf6f-5fdb218af768 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:25.022648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e25f7-7c64-4cdf-bf6f-5fdb218af768 service glance] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:25.042174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 746/1475] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:27.028788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef7d54-8a91-4a84-b676-dbbd030a70c6 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:27.029365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef7d54-8a91-4a84-b676-dbbd030a70c6 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:27.029365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef7d54-8a91-4a84-b676-dbbd030a70c6 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:27.030741 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 730/1476] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:27:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:27.036294 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a81a4-f129-4634-830c-1a0dfb2c9504 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:27.036768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a81a4-f129-4634-830c-1a0dfb2c9504 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:27.036875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a81a4-f129-4634-830c-1a0dfb2c9504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:27.109572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6f8a81a4-f129-4634-830c-1a0dfb2c9504 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:27.110853 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6f8a81a4-f129-4634-830c-1a0dfb2c9504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:27.229670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 747/1477] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:27.507334 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef7d54-8a91-4a84-b676-dbbd030a70c6 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:27.508933 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-22ef7d54-8a91-4a84-b676-dbbd030a70c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:27.635948 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef7d54-8a91-4a84-b676-dbbd030a70c6 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:27.683813 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:27.728308 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb85172d-6409-4ab3-a63c-12e6def0eebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:27.728905 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85172d-6409-4ab3-a63c-12e6def0eebf admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:27.729179 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85172d-6409-4ab3-a63c-12e6def0eebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:27.729445 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb85172d-6409-4ab3-a63c-12e6def0eebf admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:27.730095 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85172d-6409-4ab3-a63c-12e6def0eebf admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:27.731845 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: WARNING py.warnings [None req-cb85172d-6409-4ab3-a63c-12e6def0eebf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:27:27.731845 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: warnings.warn(msg) Aug 24 12:27:27.731845 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: Aug 24 12:27:27.732986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb85172d-6409-4ab3-a63c-12e6def0eebf admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:27.751701 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 731/1478] 10.222.0.58 () {56 vars in 1108 bytes} [Wed Aug 24 12:27:27 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:27:29.673375 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327933e-9c58-4138-bbd5-c55e863800e0 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:29.673957 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327933e-9c58-4138-bbd5-c55e863800e0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:29.673957 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327933e-9c58-4138-bbd5-c55e863800e0 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:29.676274 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 748/1479] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:27:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:29.682165 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea841cd0-a78b-402d-a9bd-03a375c951a0 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:29.682571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea841cd0-a78b-402d-a9bd-03a375c951a0 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:29.682874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea841cd0-a78b-402d-a9bd-03a375c951a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:29.749691 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-ea841cd0-a78b-402d-a9bd-03a375c951a0 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:29.750896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-ea841cd0-a78b-402d-a9bd-03a375c951a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:29.846697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 732/1480] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:30.137501 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327933e-9c58-4138-bbd5-c55e863800e0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:30.139116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4327933e-9c58-4138-bbd5-c55e863800e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:30.148944 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:30.287977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327933e-9c58-4138-bbd5-c55e863800e0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:30.372568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:30.406647 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f006f-c1cc-45b4-b2b7-e4afd6f059b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:30.407179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f006f-c1cc-45b4-b2b7-e4afd6f059b5 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:30.407456 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f006f-c1cc-45b4-b2b7-e4afd6f059b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:30.407759 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f006f-c1cc-45b4-b2b7-e4afd6f059b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:30.409349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e61f006f-c1cc-45b4-b2b7-e4afd6f059b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:30.498043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f006f-c1cc-45b4-b2b7-e4afd6f059b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:30.499650 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f006f-c1cc-45b4-b2b7-e4afd6f059b5 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:30.517284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 749/1481] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:32.622170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edb0e8-b311-4a1b-af61-9132df0b2f2a None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:32.622728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edb0e8-b311-4a1b-af61-9132df0b2f2a None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:32.622728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edb0e8-b311-4a1b-af61-9132df0b2f2a None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:32.624129 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 733/1482] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:27:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:32.629512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d8cab-39f0-424d-95b1-4d859b22b297 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:32.629943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d8cab-39f0-424d-95b1-4d859b22b297 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:32.630072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d8cab-39f0-424d-95b1-4d859b22b297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:32.696596 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-747d8cab-39f0-424d-95b1-4d859b22b297 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:32.697812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-747d8cab-39f0-424d-95b1-4d859b22b297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:32.746174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:32.797243 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 750/1483] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:27:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:33.082304 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edb0e8-b311-4a1b-af61-9132df0b2f2a None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:33.083874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f2edb0e8-b311-4a1b-af61-9132df0b2f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:33.107826 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:33.192346 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edb0e8-b311-4a1b-af61-9132df0b2f2a None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:33.269633 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f58dd0c-0ea3-443c-b249-1cae20515a6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:33.270234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58dd0c-0ea3-443c-b249-1cae20515a6a service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:33.270512 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58dd0c-0ea3-443c-b249-1cae20515a6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:33.270781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f58dd0c-0ea3-443c-b249-1cae20515a6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:33.272406 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7f58dd0c-0ea3-443c-b249-1cae20515a6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:33.346327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:33.380441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58dd0c-0ea3-443c-b249-1cae20515a6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:33.381873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f58dd0c-0ea3-443c-b249-1cae20515a6a service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:33.399968 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 734/1484] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:35.427391 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4441cb4-d456-4b6c-891a-8b382597af01 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:35.428031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4441cb4-d456-4b6c-891a-8b382597af01 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:35.428031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4441cb4-d456-4b6c-891a-8b382597af01 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:35.429782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 751/1485] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:35.436220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4982c-0318-4ddd-aa81-bead9138cc90 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:35.436725 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4982c-0318-4ddd-aa81-bead9138cc90 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:35.436851 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f4982c-0318-4ddd-aa81-bead9138cc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:35.516761 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-89f4982c-0318-4ddd-aa81-bead9138cc90 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:35.517986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-89f4982c-0318-4ddd-aa81-bead9138cc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:35.628950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 735/1486] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:36.000251 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4441cb4-d456-4b6c-891a-8b382597af01 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:36.001885 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b4441cb4-d456-4b6c-891a-8b382597af01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:36.133289 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4441cb4-d456-4b6c-891a-8b382597af01 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:36.208356 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:36.210139 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c662a7-ce66-4006-a64b-21bb86593354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:36.210754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c662a7-ce66-4006-a64b-21bb86593354 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:36.211225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c662a7-ce66-4006-a64b-21bb86593354 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:36.211687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c662a7-ce66-4006-a64b-21bb86593354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:36.213522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-58c662a7-ce66-4006-a64b-21bb86593354 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:36.324444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c662a7-ce66-4006-a64b-21bb86593354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:36.325909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c662a7-ce66-4006-a64b-21bb86593354 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:36.346352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 752/1487] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:38.370315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a44558-96a1-435c-9014-1f2da69cd861 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:38.370876 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a44558-96a1-435c-9014-1f2da69cd861 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:38.371007 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a44558-96a1-435c-9014-1f2da69cd861 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:38.372427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 736/1488] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:38.377657 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f8620-74e8-4088-92d5-36a618664d24 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:38.378088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f8620-74e8-4088-92d5-36a618664d24 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:38.378222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f8620-74e8-4088-92d5-36a618664d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:38.443552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-559f8620-74e8-4088-92d5-36a618664d24 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:38.445094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-559f8620-74e8-4088-92d5-36a618664d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:38.496891 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:38.542166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 753/1489] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:38.835329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a44558-96a1-435c-9014-1f2da69cd861 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:38.836982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e1a44558-96a1-435c-9014-1f2da69cd861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:38.873673 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:38.944918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a44558-96a1-435c-9014-1f2da69cd861 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:39.018009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d46252-adec-4e79-bdbc-c34091c0548e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:39.018607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d46252-adec-4e79-bdbc-c34091c0548e service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:39.018888 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d46252-adec-4e79-bdbc-c34091c0548e service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:39.019160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d46252-adec-4e79-bdbc-c34091c0548e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:39.020781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f2d46252-adec-4e79-bdbc-c34091c0548e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:39.107072 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:39.135871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d46252-adec-4e79-bdbc-c34091c0548e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:39.137828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d46252-adec-4e79-bdbc-c34091c0548e service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:39.156196 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 737/1490] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:41.329547 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372ed03-70f1-4ff2-8efc-3e3f53b32f49 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:41.330137 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372ed03-70f1-4ff2-8efc-3e3f53b32f49 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:41.330263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372ed03-70f1-4ff2-8efc-3e3f53b32f49 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:41.331591 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 754/1491] 10.222.0.58 () {54 vars in 927 bytes} [Wed Aug 24 12:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:27:41.337251 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef72094-f620-4c8b-ac1b-307536263a07 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:41.337714 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef72094-f620-4c8b-ac1b-307536263a07 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:41.337818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef72094-f620-4c8b-ac1b-307536263a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:41.404004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-bef72094-f620-4c8b-ac1b-307536263a07 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:27:41.405207 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-bef72094-f620-4c8b-ac1b-307536263a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:41.503891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 738/1492] 10.222.0.58 () {58 vars in 1024 bytes} [Wed Aug 24 12:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:27:41.798841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372ed03-70f1-4ff2-8efc-3e3f53b32f49 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:27:41.800528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8372ed03-70f1-4ff2-8efc-3e3f53b32f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:41.910616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372ed03-70f1-4ff2-8efc-3e3f53b32f49 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:27:41.984269 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e43cf6-6f67-4141-9811-46512d49d98f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:27:41.984875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e43cf6-6f67-4141-9811-46512d49d98f service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:27:41.985153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e43cf6-6f67-4141-9811-46512d49d98f service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:27:41.985424 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e43cf6-6f67-4141-9811-46512d49d98f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:27:41.987017 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-83e43cf6-6f67-4141-9811-46512d49d98f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:27:41.996479 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:42.095796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e43cf6-6f67-4141-9811-46512d49d98f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:27:42.097238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e43cf6-6f67-4141-9811-46512d49d98f service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:27:42.115063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 755/1493] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:27:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:27:51.502057 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=82 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:27:52.112840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=50 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:28:59.775064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594051a-41a4-477a-9c0d-a953db4f549e None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:59.775681 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594051a-41a4-477a-9c0d-a953db4f549e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:59.775681 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594051a-41a4-477a-9c0d-a953db4f549e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:28:59.851951 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-2594051a-41a4-477a-9c0d-a953db4f549e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:28:59.853164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2594051a-41a4-477a-9c0d-a953db4f549e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:28:59.951294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 739/1494] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:28:59.959358 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5658dd-cb1f-452d-97ef-61cdb543821a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:28:59.959866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5658dd-cb1f-452d-97ef-61cdb543821a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:28:59.959983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5658dd-cb1f-452d-97ef-61cdb543821a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:00.037874 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5f5658dd-cb1f-452d-97ef-61cdb543821a None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:00.039072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5f5658dd-cb1f-452d-97ef-61cdb543821a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:00.134021 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 756/1495] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:00.139444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594051a-41a4-477a-9c0d-a953db4f549e None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:00.140698 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2594051a-41a4-477a-9c0d-a953db4f549e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:00.152079 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594051a-41a4-477a-9c0d-a953db4f549e None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:00.212543 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:00.246957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb8a9a-19cd-4da8-b759-6a43a7eda806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:00.247554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb8a9a-19cd-4da8-b759-6a43a7eda806 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:00.247862 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb8a9a-19cd-4da8-b759-6a43a7eda806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:00.248131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb8a9a-19cd-4da8-b759-6a43a7eda806 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:00.248758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb8a9a-19cd-4da8-b759-6a43a7eda806 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:29:00.249053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb8a9a-19cd-4da8-b759-6a43a7eda806 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:00.250066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: WARNING py.warnings [None req-cabb8a9a-19cd-4da8-b759-6a43a7eda806 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:00.250066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: warnings.warn(msg) Aug 24 12:29:00.250066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: Aug 24 12:29:00.250947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb8a9a-19cd-4da8-b759-6a43a7eda806 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:00.262390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 740/1496] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:29:00.272140 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5658dd-cb1f-452d-97ef-61cdb543821a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:00.281764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5658dd-cb1f-452d-97ef-61cdb543821a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:00.347061 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:00.382938 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a02b70-8e3d-4db5-be9e-91544b760c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:00.383551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a02b70-8e3d-4db5-be9e-91544b760c03 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:00.383875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a02b70-8e3d-4db5-be9e-91544b760c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:00.384198 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a02b70-8e3d-4db5-be9e-91544b760c03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:00.384775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a02b70-8e3d-4db5-be9e-91544b760c03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:00.387265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-60a02b70-8e3d-4db5-be9e-91544b760c03 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:00.387265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 24 12:29:00.387265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: Aug 24 12:29:00.388510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a02b70-8e3d-4db5-be9e-91544b760c03 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:00.444471 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 757/1497] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:29:00 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:00.449399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb8a9a-19cd-4da8-b759-6a43a7eda806 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:00.451194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-cabb8a9a-19cd-4da8-b759-6a43a7eda806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:00.485370 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb8a9a-19cd-4da8-b759-6a43a7eda806 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:00.580017 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62292e9c-7ccc-4c3c-9411-441cf00abb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:00.580622 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62292e9c-7ccc-4c3c-9411-441cf00abb87 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:00.580901 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62292e9c-7ccc-4c3c-9411-441cf00abb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:00.581173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62292e9c-7ccc-4c3c-9411-441cf00abb87 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:00.581790 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62292e9c-7ccc-4c3c-9411-441cf00abb87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:00.582811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: WARNING py.warnings [None req-62292e9c-7ccc-4c3c-9411-441cf00abb87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:00.582811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: warnings.warn(msg) Aug 24 12:29:00.582811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: Aug 24 12:29:00.584059 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62292e9c-7ccc-4c3c-9411-441cf00abb87 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:00.625987 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 741/1498] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:29:00 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:29:00.630224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a02b70-8e3d-4db5-be9e-91544b760c03 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:00.631787 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-60a02b70-8e3d-4db5-be9e-91544b760c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:00.692563 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a02b70-8e3d-4db5-be9e-91544b760c03 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:00.779721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:00.790749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd19d12-a749-432e-80ef-a0d890c9dac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:00.791353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd19d12-a749-432e-80ef-a0d890c9dac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:00.791649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd19d12-a749-432e-80ef-a0d890c9dac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:00.791929 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd19d12-a749-432e-80ef-a0d890c9dac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:00.792645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd19d12-a749-432e-80ef-a0d890c9dac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:00.793670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-cbd19d12-a749-432e-80ef-a0d890c9dac7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:00.793670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 24 12:29:00.793670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: Aug 24 12:29:00.794562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd19d12-a749-432e-80ef-a0d890c9dac7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:00.806905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 758/1499] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:00.810889 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62292e9c-7ccc-4c3c-9411-441cf00abb87 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:00.812429 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-62292e9c-7ccc-4c3c-9411-441cf00abb87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:00.824211 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62292e9c-7ccc-4c3c-9411-441cf00abb87 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:00.850014 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:00.920250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d30835-535b-4131-bb80-4802f469e7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:00.920856 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d30835-535b-4131-bb80-4802f469e7bf admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:00.921131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d30835-535b-4131-bb80-4802f469e7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:00.921398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d30835-535b-4131-bb80-4802f469e7bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:00.922033 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d30835-535b-4131-bb80-4802f469e7bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:00.923079 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: WARNING py.warnings [None req-b5d30835-535b-4131-bb80-4802f469e7bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:00.923079 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: warnings.warn(msg) Aug 24 12:29:00.923079 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: Aug 24 12:29:00.923988 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d30835-535b-4131-bb80-4802f469e7bf admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:00.936861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 742/1500] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:29:00.941100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd19d12-a749-432e-80ef-a0d890c9dac7 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:00.942751 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cbd19d12-a749-432e-80ef-a0d890c9dac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:00.976787 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd19d12-a749-432e-80ef-a0d890c9dac7 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:01.072290 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7cd95c-f24f-4e13-8565-21cb600ca787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:01.072976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cd95c-f24f-4e13-8565-21cb600ca787 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:01.073307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cd95c-f24f-4e13-8565-21cb600ca787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:01.074119 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cd95c-f24f-4e13-8565-21cb600ca787 admin admin] PATH_INFO: `/v3/projects/9b38e46626354fffb149d75901d0a295/users/91c6cd1c2feb4d09a98cb76495864e01/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:01.095946 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7cd95c-f24f-4e13-8565-21cb600ca787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b38e46626354fffb149d75901d0a295, user_id=91c6cd1c2feb4d09a98cb76495864e01, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:01.096976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-0a7cd95c-f24f-4e13-8565-21cb600ca787 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:01.096976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 24 12:29:01.096976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: Aug 24 12:29:01.098936 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7cd95c-f24f-4e13-8565-21cb600ca787 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:01.123507 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 759/1501] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:29:00 2022] PUT /identity/v3/projects/9b38e46626354fffb149d75901d0a295/users/91c6cd1c2feb4d09a98cb76495864e01/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:01.136487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce5858-208e-43b4-a5b1-6d6e518a61b9 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:01.136874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce5858-208e-43b4-a5b1-6d6e518a61b9 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:01.137198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dce5858-208e-43b4-a5b1-6d6e518a61b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:01.193682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-8dce5858-208e-43b4-a5b1-6d6e518a61b9 None None] MFA Rules not processed for user `91c6cd1c2feb4d09a98cb76495864e01`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:01.194823 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-8dce5858-208e-43b4-a5b1-6d6e518a61b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:01.272224 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 743/1502] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:29:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:01.285529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae6a56-4569-4793-a977-2ca879ccb3b9 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:01.285822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae6a56-4569-4793-a977-2ca879ccb3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:01.286103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae6a56-4569-4793-a977-2ca879ccb3b9 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:01.287474 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 760/1503] 10.222.0.58 () {54 vars in 964 bytes} [Wed Aug 24 12:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:29:01.292753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaf092-88a0-42f8-b2ee-42d967d2cc6f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:01.293037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaf092-88a0-42f8-b2ee-42d967d2cc6f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:01.293311 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaf092-88a0-42f8-b2ee-42d967d2cc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:01.329912 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:01.374534 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-caaaf092-88a0-42f8-b2ee-42d967d2cc6f None None] MFA Rules not processed for user `a42d8e462ba542a5bf58ba052dceeb8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:01.375792 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-caaaf092-88a0-42f8-b2ee-42d967d2cc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:01.449541 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 744/1504] 10.222.0.58 () {58 vars in 1061 bytes} [Wed Aug 24 12:29:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:01.457413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae6a56-4569-4793-a977-2ca879ccb3b9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:01.458927 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-93ae6a56-4569-4793-a977-2ca879ccb3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:01.509176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:01.564254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae6a56-4569-4793-a977-2ca879ccb3b9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:01.644102 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de15f03-c55c-4254-b7e2-e38aa01e0011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:01.644708 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de15f03-c55c-4254-b7e2-e38aa01e0011 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:01.644985 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de15f03-c55c-4254-b7e2-e38aa01e0011 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:01.645251 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de15f03-c55c-4254-b7e2-e38aa01e0011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:01.646837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0de15f03-c55c-4254-b7e2-e38aa01e0011 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:01.748812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de15f03-c55c-4254-b7e2-e38aa01e0011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:01.750170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de15f03-c55c-4254-b7e2-e38aa01e0011 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:01.753683 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:01.768677 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 761/1505] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:01.814606 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ca1ae-c6f8-419d-8768-ceee15c80ee1 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:01.815028 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ca1ae-c6f8-419d-8768-ceee15c80ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:01.815161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ca1ae-c6f8-419d-8768-ceee15c80ee1 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:01.816477 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 745/1506] 10.222.0.58 () {52 vars in 830 bytes} [Wed Aug 24 12:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:29:01.820149 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de15f03-c55c-4254-b7e2-e38aa01e0011 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:01.821361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0de15f03-c55c-4254-b7e2-e38aa01e0011 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:01.922941 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de15f03-c55c-4254-b7e2-e38aa01e0011 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:02.015425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66102b2-71d2-48b0-a26c-015a4430ac8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:02.016109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66102b2-71d2-48b0-a26c-015a4430ac8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:02.016292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66102b2-71d2-48b0-a26c-015a4430ac8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:02.016561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66102b2-71d2-48b0-a26c-015a4430ac8a admin admin] PATH_INFO: `/v3/users/91c6cd1c2feb4d09a98cb76495864e01` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:02.029606 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66102b2-71d2-48b0-a26c-015a4430ac8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=91c6cd1c2feb4d09a98cb76495864e01)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:02.030631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-c66102b2-71d2-48b0-a26c-015a4430ac8a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:02.030631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 24 12:29:02.030631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: Aug 24 12:29:02.031908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66102b2-71d2-48b0-a26c-015a4430ac8a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:02.041545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:02.099760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c66102b2-71d2-48b0-a26c-015a4430ac8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91c6cd1c2feb4d09a98cb76495864e01'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:02.100341 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c66102b2-71d2-48b0-a26c-015a4430ac8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91c6cd1c2feb4d09a98cb76495864e01'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:02.114471 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c66102b2-71d2-48b0-a26c-015a4430ac8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91c6cd1c2feb4d09a98cb76495864e01'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:02.123665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c66102b2-71d2-48b0-a26c-015a4430ac8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91c6cd1c2feb4d09a98cb76495864e01'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:02.146420 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c66102b2-71d2-48b0-a26c-015a4430ac8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91c6cd1c2feb4d09a98cb76495864e01'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:02.152536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 762/1507] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:29:01 2022] DELETE /identity/v3/users/91c6cd1c2feb4d09a98cb76495864e01 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:02.157016 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065ff2d-63b2-4557-8b7a-99f32733444f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:02.157317 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065ff2d-63b2-4557-8b7a-99f32733444f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:02.157586 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065ff2d-63b2-4557-8b7a-99f32733444f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:02.215269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-f065ff2d-63b2-4557-8b7a-99f32733444f None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:29:02.216518 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f065ff2d-63b2-4557-8b7a-99f32733444f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:02.291284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:02.304352 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 746/1508] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:29:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:29:02.314398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66102b2-71d2-48b0-a26c-015a4430ac8a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:02.315462 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c66102b2-71d2-48b0-a26c-015a4430ac8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:02.421549 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66102b2-71d2-48b0-a26c-015a4430ac8a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:02.476185 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:02.497166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b57a8-2eae-4d3a-8dba-e67b6e26a504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:02.497783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b57a8-2eae-4d3a-8dba-e67b6e26a504 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:02.498067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b57a8-2eae-4d3a-8dba-e67b6e26a504 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:02.498344 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b57a8-2eae-4d3a-8dba-e67b6e26a504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:02.500015 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c23b57a8-2eae-4d3a-8dba-e67b6e26a504 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:02.611756 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b57a8-2eae-4d3a-8dba-e67b6e26a504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:02.613209 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b57a8-2eae-4d3a-8dba-e67b6e26a504 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:02.631775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 763/1509] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:29:02.980968 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065ff2d-63b2-4557-8b7a-99f32733444f None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:29:02.982257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f065ff2d-63b2-4557-8b7a-99f32733444f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:29:02.993628 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065ff2d-63b2-4557-8b7a-99f32733444f None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:29:03.093746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78be64-a061-4bde-92fa-a53be1bd909b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:29:03.094377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78be64-a061-4bde-92fa-a53be1bd909b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:29:03.094657 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78be64-a061-4bde-92fa-a53be1bd909b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:29:03.094925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78be64-a061-4bde-92fa-a53be1bd909b admin admin] PATH_INFO: `/v3/projects/9b38e46626354fffb149d75901d0a295` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:29:03.104669 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78be64-a061-4bde-92fa-a53be1bd909b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b38e46626354fffb149d75901d0a295)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:29:03.105691 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: WARNING py.warnings [None req-da78be64-a061-4bde-92fa-a53be1bd909b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:29:03.105691 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: warnings.warn(msg) Aug 24 12:29:03.105691 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: Aug 24 12:29:03.106894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78be64-a061-4bde-92fa-a53be1bd909b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:29:03.163879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-da78be64-a061-4bde-92fa-a53be1bd909b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b38e46626354fffb149d75901d0a295', 'request_id': 'req-da78be64-a061-4bde-92fa-a53be1bd909b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:03.169926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-da78be64-a061-4bde-92fa-a53be1bd909b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b38e46626354fffb149d75901d0a295', 'request_id': 'req-da78be64-a061-4bde-92fa-a53be1bd909b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:03.178996 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-da78be64-a061-4bde-92fa-a53be1bd909b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b38e46626354fffb149d75901d0a295', 'request_id': 'req-da78be64-a061-4bde-92fa-a53be1bd909b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:03.209191 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-da78be64-a061-4bde-92fa-a53be1bd909b admin admin] The token cache is being invalidate because project 9b38e46626354fffb149d75901d0a295 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:29:03.209473 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-da78be64-a061-4bde-92fa-a53be1bd909b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:03.209985 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-da78be64-a061-4bde-92fa-a53be1bd909b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:29:03.211287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 747/1510] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:29:02 2022] DELETE /identity/v3/projects/9b38e46626354fffb149d75901d0a295 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:29:12.629497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=62 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:29:13.207212 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:28.353920 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8c8c9-2d75-46ff-af08-f28a406b821c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:28.354530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8c8c9-2d75-46ff-af08-f28a406b821c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:28.354530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8c8c9-2d75-46ff-af08-f28a406b821c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:28.355972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:28.362005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de711d1-3785-495f-bc05-0dc0a97201ee None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:28.362476 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de711d1-3785-495f-bc05-0dc0a97201ee None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:28.362614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de711d1-3785-495f-bc05-0dc0a97201ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:28.443204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-0de711d1-3785-495f-bc05-0dc0a97201ee None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:28.444465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-0de711d1-3785-495f-bc05-0dc0a97201ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:28.553273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:28.712767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8c8c9-2d75-46ff-af08-f28a406b821c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:28.715011 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b3e8c8c9-2d75-46ff-af08-f28a406b821c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:28.967884 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8c8c9-2d75-46ff-af08-f28a406b821c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:28.973345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68bf59-8776-4205-90ce-76ef1fff2bd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:28.973947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68bf59-8776-4205-90ce-76ef1fff2bd2 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:28.974223 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68bf59-8776-4205-90ce-76ef1fff2bd2 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:28.974487 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68bf59-8776-4205-90ce-76ef1fff2bd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:28.983484 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68bf59-8776-4205-90ce-76ef1fff2bd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:28.985146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68bf59-8776-4205-90ce-76ef1fff2bd2 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:28.994751 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 765/1513] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:29.844984 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd459d2-8931-4ec8-bc71-63fd1e1b9ee5 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:29.845524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd459d2-8931-4ec8-bc71-63fd1e1b9ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:29.845737 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd459d2-8931-4ec8-bc71-63fd1e1b9ee5 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:29.847086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:29.852779 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67164ff6-e857-4d0f-b0aa-6b2781fdd39d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:29.853275 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67164ff6-e857-4d0f-b0aa-6b2781fdd39d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:29.853572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67164ff6-e857-4d0f-b0aa-6b2781fdd39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:29.874423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-67164ff6-e857-4d0f-b0aa-6b2781fdd39d None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:29.875775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-67164ff6-e857-4d0f-b0aa-6b2781fdd39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:29.888510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:30:30.045915 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd459d2-8931-4ec8-bc71-63fd1e1b9ee5 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:30.061992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd459d2-8931-4ec8-bc71-63fd1e1b9ee5 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:30.068007 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1f806-caed-48fb-b634-589dfd38ce08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:30.068956 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1f806-caed-48fb-b634-589dfd38ce08 service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:30.069614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1f806-caed-48fb-b634-589dfd38ce08 service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:30.070152 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1f806-caed-48fb-b634-589dfd38ce08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:30.081808 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1f806-caed-48fb-b634-589dfd38ce08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:30.084899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1f806-caed-48fb-b634-589dfd38ce08 service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:30.097446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 750/1516] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:30.300973 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891886f-3648-4772-997c-f3f7bcdee7d6 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:30.301260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891886f-3648-4772-997c-f3f7bcdee7d6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:30.301538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891886f-3648-4772-997c-f3f7bcdee7d6 None None] PATH_INFO: `/v3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:30.302828 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 767/1517] 10.222.0.57 () {54 vars in 925 bytes} [Wed Aug 24 12:30:30 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:30:30.309245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d335c75-815e-4b4c-a55a-c79b40534c29 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:30.309643 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d335c75-815e-4b4c-a55a-c79b40534c29 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:30.309983 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d335c75-815e-4b4c-a55a-c79b40534c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:30.381636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-2d335c75-815e-4b4c-a55a-c79b40534c29 None None] MFA Rules not processed for user `b7ba45e3cff74962965b248f9f017f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:30.382869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2d335c75-815e-4b4c-a55a-c79b40534c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:30.405606 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:30.464451 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 751/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Wed Aug 24 12:30:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:34.280284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885079f-d753-484c-976d-996d14c15167 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:34.280791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885079f-d753-484c-976d-996d14c15167 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:34.280791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d885079f-d753-484c-976d-996d14c15167 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:34.282272 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 768/1519] 10.222.0.58 () {54 vars in 961 bytes} [Wed Aug 24 12:30:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:34.287653 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f6d57-932e-4f18-bcec-bd959df7815b None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:34.288126 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f6d57-932e-4f18-bcec-bd959df7815b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:34.288223 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f6d57-932e-4f18-bcec-bd959df7815b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:34.308645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-088f6d57-932e-4f18-bcec-bd959df7815b None None] MFA Rules not processed for user `b7ba45e3cff74962965b248f9f017f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:34.309842 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-088f6d57-932e-4f18-bcec-bd959df7815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:34.319723 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 752/1520] 10.222.0.58 () {58 vars in 1058 bytes} [Wed Aug 24 12:30:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:34.328003 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885079f-d753-484c-976d-996d14c15167 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:34.341481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d885079f-d753-484c-976d-996d14c15167 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:34.346506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155eeb6-2191-4c27-99a2-9452da860989 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ba45e3cff74962965b248f9f017f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:34.347115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155eeb6-2191-4c27-99a2-9452da860989 service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:34.347382 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155eeb6-2191-4c27-99a2-9452da860989 service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:34.347730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155eeb6-2191-4c27-99a2-9452da860989 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:34.356263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155eeb6-2191-4c27-99a2-9452da860989 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:34.358385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0155eeb6-2191-4c27-99a2-9452da860989 service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:34.368628 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 769/1521] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:41.332690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953e8c9-3c77-4187-a348-5d16e997d5d7 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:41.333247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953e8c9-3c77-4187-a348-5d16e997d5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:41.333247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e953e8c9-3c77-4187-a348-5d16e997d5d7 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:41.334618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:41.340006 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b511e-131b-40aa-bc48-a739de9771fa None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:41.340412 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b511e-131b-40aa-bc48-a739de9771fa None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:41.340568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b511e-131b-40aa-bc48-a739de9771fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:41.361248 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b78b511e-131b-40aa-bc48-a739de9771fa None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:41.362438 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b78b511e-131b-40aa-bc48-a739de9771fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:41.370321 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:41.529986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953e8c9-3c77-4187-a348-5d16e997d5d7 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:41.544368 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e953e8c9-3c77-4187-a348-5d16e997d5d7 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:41.550422 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4b85b-2852-4dc2-8228-9315ec113fc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:41.551017 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4b85b-2852-4dc2-8228-9315ec113fc2 service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:41.551301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4b85b-2852-4dc2-8228-9315ec113fc2 service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:41.551567 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4b85b-2852-4dc2-8228-9315ec113fc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:41.559414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4b85b-2852-4dc2-8228-9315ec113fc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:41.561023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4b85b-2852-4dc2-8228-9315ec113fc2 service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:41.571062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 754/1524] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:30:42.397044 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fae829-126c-479e-b485-58bea5728bdc None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:42.397588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fae829-126c-479e-b485-58bea5728bdc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:42.397588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fae829-126c-479e-b485-58bea5728bdc None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:42.398985 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 12:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:42.403936 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859c29e-ed67-408e-ab11-1b751a805156 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:42.404366 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859c29e-ed67-408e-ab11-1b751a805156 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:42.404472 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859c29e-ed67-408e-ab11-1b751a805156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:42.423761 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-e859c29e-ed67-408e-ab11-1b751a805156 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:42.424948 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e859c29e-ed67-408e-ab11-1b751a805156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:42.432656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 12:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:42.589854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fae829-126c-479e-b485-58bea5728bdc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:42.604926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fae829-126c-479e-b485-58bea5728bdc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:42.610338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623da2c4-4052-44a2-bf4d-45ff09b48927 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:42.610926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623da2c4-4052-44a2-bf4d-45ff09b48927 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:42.611207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623da2c4-4052-44a2-bf4d-45ff09b48927 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:42.611523 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623da2c4-4052-44a2-bf4d-45ff09b48927 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:42.619485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623da2c4-4052-44a2-bf4d-45ff09b48927 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:42.621112 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623da2c4-4052-44a2-bf4d-45ff09b48927 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:42.630416 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 772/1527] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:46.710576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc752971-be33-48d0-bf4b-a40aad69e26c None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:46.711187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc752971-be33-48d0-bf4b-a40aad69e26c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:46.711187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc752971-be33-48d0-bf4b-a40aad69e26c None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:46.712553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:30:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:46.717638 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0259248-e995-471e-b4f6-09841f10993f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:46.718115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0259248-e995-471e-b4f6-09841f10993f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:46.718220 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0259248-e995-471e-b4f6-09841f10993f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:46.738675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f0259248-e995-471e-b4f6-09841f10993f None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:46.739916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f0259248-e995-471e-b4f6-09841f10993f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:46.747783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:30:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:47.035776 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc752971-be33-48d0-bf4b-a40aad69e26c None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:47.037465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-dc752971-be33-48d0-bf4b-a40aad69e26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:47.114781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc752971-be33-48d0-bf4b-a40aad69e26c None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:47.118351 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53539df4-4c3b-48aa-a03a-4f5a77a9ff83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:47.118956 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53539df4-4c3b-48aa-a03a-4f5a77a9ff83 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:47.119235 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53539df4-4c3b-48aa-a03a-4f5a77a9ff83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:47.119506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53539df4-4c3b-48aa-a03a-4f5a77a9ff83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:47.128910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53539df4-4c3b-48aa-a03a-4f5a77a9ff83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:47.130337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53539df4-4c3b-48aa-a03a-4f5a77a9ff83 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:47.139503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 757/1530] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:49.042443 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e66455-9f79-4fc4-b34a-7eafe2d4f80c None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:49.042999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e66455-9f79-4fc4-b34a-7eafe2d4f80c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:49.042999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e66455-9f79-4fc4-b34a-7eafe2d4f80c None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:49.044422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:49.050311 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7df908-2ac8-459b-85e6-574be2a8299b None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:49.050745 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7df908-2ac8-459b-85e6-574be2a8299b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:49.050848 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7df908-2ac8-459b-85e6-574be2a8299b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:49.069925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-4b7df908-2ac8-459b-85e6-574be2a8299b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:49.071110 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-4b7df908-2ac8-459b-85e6-574be2a8299b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:49.079642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:49.366459 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e66455-9f79-4fc4-b34a-7eafe2d4f80c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:49.368196 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-34e66455-9f79-4fc4-b34a-7eafe2d4f80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:49.381034 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e66455-9f79-4fc4-b34a-7eafe2d4f80c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:49.384595 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b953aa1-81bc-48e8-92e6-c0a2f6ee8ab9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:49.385176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b953aa1-81bc-48e8-92e6-c0a2f6ee8ab9 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:49.385453 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b953aa1-81bc-48e8-92e6-c0a2f6ee8ab9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:49.385723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b953aa1-81bc-48e8-92e6-c0a2f6ee8ab9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:49.395047 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b953aa1-81bc-48e8-92e6-c0a2f6ee8ab9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:49.396491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b953aa1-81bc-48e8-92e6-c0a2f6ee8ab9 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:49.405678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 775/1533] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:52.809498 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187d567-5fe0-4dd2-a3ce-68b466a0715b None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:52.810050 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187d567-5fe0-4dd2-a3ce-68b466a0715b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:52.810050 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187d567-5fe0-4dd2-a3ce-68b466a0715b None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:52.811452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:52.816767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74eebd6-126c-4aa1-a5bd-0c1d95c636c4 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:52.817169 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74eebd6-126c-4aa1-a5bd-0c1d95c636c4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:52.817320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74eebd6-126c-4aa1-a5bd-0c1d95c636c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:52.824752 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=81 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:52.839747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a74eebd6-126c-4aa1-a5bd-0c1d95c636c4 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:52.840961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a74eebd6-126c-4aa1-a5bd-0c1d95c636c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:52.851579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:30:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:53.406961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187d567-5fe0-4dd2-a3ce-68b466a0715b None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:53.408670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-8187d567-5fe0-4dd2-a3ce-68b466a0715b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:53.473861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:30:53.487674 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187d567-5fe0-4dd2-a3ce-68b466a0715b None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:53.491208 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae0659-fe53-47a4-a58b-9493e2d53ebe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:53.491836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae0659-fe53-47a4-a58b-9493e2d53ebe service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:53.492119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae0659-fe53-47a4-a58b-9493e2d53ebe service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:53.492389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae0659-fe53-47a4-a58b-9493e2d53ebe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:53.501962 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae0659-fe53-47a4-a58b-9493e2d53ebe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:53.503383 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae0659-fe53-47a4-a58b-9493e2d53ebe service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:53.512745 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 760/1536] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:30:55.438778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eef946-de83-4c31-9653-52771641172a None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:55.439348 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eef946-de83-4c31-9653-52771641172a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:55.439348 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eef946-de83-4c31-9653-52771641172a None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:55.440705 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 12:30:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:30:55.446060 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689bacb-d119-43df-b997-34864e666686 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:55.446509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689bacb-d119-43df-b997-34864e666686 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:55.446647 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1689bacb-d119-43df-b997-34864e666686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:55.466627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-1689bacb-d119-43df-b997-34864e666686 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:30:55.467868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1689bacb-d119-43df-b997-34864e666686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:55.476340 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 12:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:30:56.030623 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eef946-de83-4c31-9653-52771641172a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:30:56.032931 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b0eef946-de83-4c31-9653-52771641172a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:30:56.045847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eef946-de83-4c31-9653-52771641172a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:30:56.049389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dbd4d-3cdd-47ab-96af-559ab120a9c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:30:56.049977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dbd4d-3cdd-47ab-96af-559ab120a9c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:30:56.050252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dbd4d-3cdd-47ab-96af-559ab120a9c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:30:56.050520 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dbd4d-3cdd-47ab-96af-559ab120a9c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:30:56.059148 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67dbd4d-3cdd-47ab-96af-559ab120a9c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:30:56.060589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67dbd4d-3cdd-47ab-96af-559ab120a9c2 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:30:56.070003 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 778/1539] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:30:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:31:05.461946 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=11 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:31:06.065932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=8 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:18.927400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d13f1d-4064-4dfa-ae47-201aafcdd122 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:18.928064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d13f1d-4064-4dfa-ae47-201aafcdd122 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:18.928064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d13f1d-4064-4dfa-ae47-201aafcdd122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:18.949850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-c3d13f1d-4064-4dfa-ae47-201aafcdd122 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:18.951092 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c3d13f1d-4064-4dfa-ae47-201aafcdd122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:18.965398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 762/1540] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:18.975924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda615f-7968-4651-b0fc-f461cf258903 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:18.977003 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda615f-7968-4651-b0fc-f461cf258903 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:18.977510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda615f-7968-4651-b0fc-f461cf258903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.008746 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-aeda615f-7968-4651-b0fc-f461cf258903 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:19.010354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-aeda615f-7968-4651-b0fc-f461cf258903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:19.025004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 779/1541] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:19.031256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d13f1d-4064-4dfa-ae47-201aafcdd122 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:19.045481 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d13f1d-4064-4dfa-ae47-201aafcdd122 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:19.056487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaf5ca-fc67-4044-b3eb-93488a6458e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:19.057149 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaf5ca-fc67-4044-b3eb-93488a6458e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.057438 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaf5ca-fc67-4044-b3eb-93488a6458e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.057718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaf5ca-fc67-4044-b3eb-93488a6458e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.058417 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abaf5ca-fc67-4044-b3eb-93488a6458e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:19.058728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abaf5ca-fc67-4044-b3eb-93488a6458e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:19.060476 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abaf5ca-fc67-4044-b3eb-93488a6458e9 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:19.083543 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 763/1542] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:19.092289 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda615f-7968-4651-b0fc-f461cf258903 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:19.106330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda615f-7968-4651-b0fc-f461cf258903 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:19.115355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229bb8d-2ae2-4b87-9603-d60b2de6b85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:19.116668 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229bb8d-2ae2-4b87-9603-d60b2de6b85c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.117385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229bb8d-2ae2-4b87-9603-d60b2de6b85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.117996 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229bb8d-2ae2-4b87-9603-d60b2de6b85c admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.119161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229bb8d-2ae2-4b87-9603-d60b2de6b85c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:19.121760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229bb8d-2ae2-4b87-9603-d60b2de6b85c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:19.187379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 780/1543] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:19.192300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaf5ca-fc67-4044-b3eb-93488a6458e9 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:19.206720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaf5ca-fc67-4044-b3eb-93488a6458e9 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:19.253121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b5cee-e752-44f8-acfc-e12498391f8c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.255663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b5cee-e752-44f8-acfc-e12498391f8c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.256821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b5cee-e752-44f8-acfc-e12498391f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.308230 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9b1b5cee-e752-44f8-acfc-e12498391f8c None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:19.309894 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9b1b5cee-e752-44f8-acfc-e12498391f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:19.311565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c81151-0d63-40fc-8038-b6d7934e2fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:19.311565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c81151-0d63-40fc-8038-b6d7934e2fed admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.311565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c81151-0d63-40fc-8038-b6d7934e2fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.311565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c81151-0d63-40fc-8038-b6d7934e2fed admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.312649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c81151-0d63-40fc-8038-b6d7934e2fed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:19.315022 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c81151-0d63-40fc-8038-b6d7934e2fed admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:19.326236 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 781/1544] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:19.338175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30ef1d-457a-4a9a-8f3a-735e39543d9b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.339906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30ef1d-457a-4a9a-8f3a-735e39543d9b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.339906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac30ef1d-457a-4a9a-8f3a-735e39543d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.354959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 764/1545] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:19.368451 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c81151-0d63-40fc-8038-b6d7934e2fed admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:19.376351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ac30ef1d-457a-4a9a-8f3a-735e39543d9b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:19.377654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ac30ef1d-457a-4a9a-8f3a-735e39543d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:19.386092 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c81151-0d63-40fc-8038-b6d7934e2fed admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:19.392770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 782/1546] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:19.394158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff55cfa-2342-4d28-aaa1-92a69d2ea32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:19.394790 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff55cfa-2342-4d28-aaa1-92a69d2ea32a admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.395075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff55cfa-2342-4d28-aaa1-92a69d2ea32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.395348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff55cfa-2342-4d28-aaa1-92a69d2ea32a admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.396030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff55cfa-2342-4d28-aaa1-92a69d2ea32a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:19.397463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30ef1d-457a-4a9a-8f3a-735e39543d9b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:19.397742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff55cfa-2342-4d28-aaa1-92a69d2ea32a admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:19.411417 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac30ef1d-457a-4a9a-8f3a-735e39543d9b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:19.419346 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 765/1547] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:32:19.422861 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ba7f5-94dc-4629-baed-5ddc41ac35d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:19.424088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ba7f5-94dc-4629-baed-5ddc41ac35d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.424490 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff55cfa-2342-4d28-aaa1-92a69d2ea32a admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:19.424922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ba7f5-94dc-4629-baed-5ddc41ac35d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.425410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ba7f5-94dc-4629-baed-5ddc41ac35d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.426370 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ba7f5-94dc-4629-baed-5ddc41ac35d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:19.427020 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ba7f5-94dc-4629-baed-5ddc41ac35d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:19.428275 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-5f8ba7f5-94dc-4629-baed-5ddc41ac35d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:32:19.428275 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 24 12:32:19.428275 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: Aug 24 12:32:19.429569 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ba7f5-94dc-4629-baed-5ddc41ac35d7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:19.442904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff55cfa-2342-4d28-aaa1-92a69d2ea32a admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:19.453347 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 783/1548] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:19.454277 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39c4b8-3281-4550-9060-409bd03f9d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:19.454916 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39c4b8-3281-4550-9060-409bd03f9d04 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.455201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39c4b8-3281-4550-9060-409bd03f9d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.455506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39c4b8-3281-4550-9060-409bd03f9d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.456154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9518886-5c1b-4bc0-b767-0debff568d17 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.456283 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39c4b8-3281-4550-9060-409bd03f9d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:19.456766 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9518886-5c1b-4bc0-b767-0debff568d17 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.457253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9518886-5c1b-4bc0-b767-0debff568d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.457838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39c4b8-3281-4550-9060-409bd03f9d04 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:19.476498 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 766/1549] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:32:19.477283 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39c4b8-3281-4550-9060-409bd03f9d04 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:19.493106 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b9518886-5c1b-4bc0-b767-0debff568d17 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:19.494326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b9518886-5c1b-4bc0-b767-0debff568d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:19.501714 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39c4b8-3281-4550-9060-409bd03f9d04 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:19.511302 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427a546-dc38-4b01-9b08-820b4329e96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:19.511955 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427a546-dc38-4b01-9b08-820b4329e96a admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.512238 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427a546-dc38-4b01-9b08-820b4329e96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.512514 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427a546-dc38-4b01-9b08-820b4329e96a admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.513145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427a546-dc38-4b01-9b08-820b4329e96a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:19.514161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: WARNING py.warnings [None req-f427a546-dc38-4b01-9b08-820b4329e96a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:32:19.514161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: warnings.warn(msg) Aug 24 12:32:19.514161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: Aug 24 12:32:19.517292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 784/1550] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:19.518568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaed76-4d03-4e61-ab86-da513984847e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.518920 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaed76-4d03-4e61-ab86-da513984847e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.519198 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaed76-4d03-4e61-ab86-da513984847e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.527008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427a546-dc38-4b01-9b08-820b4329e96a admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:19.549308 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-dadaed76-4d03-4e61-ab86-da513984847e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:19.550651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-dadaed76-4d03-4e61-ab86-da513984847e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:19.559575 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 785/1551] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:19.560424 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaed76-4d03-4e61-ab86-da513984847e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:19.560539 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 767/1552] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/projects => generated 379 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:19.564419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640f6a2-43a0-45a2-b6be-4c054b3e6da5 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.564419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640f6a2-43a0-45a2-b6be-4c054b3e6da5 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.564419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640f6a2-43a0-45a2-b6be-4c054b3e6da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.571564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaed76-4d03-4e61-ab86-da513984847e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:19.594753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-4640f6a2-43a0-45a2-b6be-4c054b3e6da5 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:19.596032 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-4640f6a2-43a0-45a2-b6be-4c054b3e6da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:19.630708 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d0376-01b1-4f8f-ac11-dc0bd2e3d575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:19.631388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d0376-01b1-4f8f-ac11-dc0bd2e3d575 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.632558 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d0376-01b1-4f8f-ac11-dc0bd2e3d575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.632558 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d0376-01b1-4f8f-ac11-dc0bd2e3d575 admin admin] PATH_INFO: `/v3/projects/7b5a8391b9c34b80b3cd6f6408c0e13b/users/8c4727a0ef41431992676f231464f041/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.646880 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d0376-01b1-4f8f-ac11-dc0bd2e3d575 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b5a8391b9c34b80b3cd6f6408c0e13b, user_id=8c4727a0ef41431992676f231464f041, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:19.649545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d0376-01b1-4f8f-ac11-dc0bd2e3d575 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:19.659290 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 768/1553] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:19.660587 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef3300-d958-4660-b57c-dd1a3d670a75 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.660872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef3300-d958-4660-b57c-dd1a3d670a75 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.661146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef3300-d958-4660-b57c-dd1a3d670a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.663389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 786/1554] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:19 2022] PUT /identity/v3/projects/7b5a8391b9c34b80b3cd6f6408c0e13b/users/8c4727a0ef41431992676f231464f041/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:19.664548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c7223-234f-497f-b56e-70b030ac98f7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.664840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c7223-234f-497f-b56e-70b030ac98f7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.665117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c7223-234f-497f-b56e-70b030ac98f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.684369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-068c7223-234f-497f-b56e-70b030ac98f7 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:19.685582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-068c7223-234f-497f-b56e-70b030ac98f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:19.690904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-86ef3300-d958-4660-b57c-dd1a3d670a75 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:19.692120 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-86ef3300-d958-4660-b57c-dd1a3d670a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:19.713738 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:19.747580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 787/1555] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:19.748863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e08815-bed0-4bc5-8a9b-4dc041396127 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.749780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e08815-bed0-4bc5-8a9b-4dc041396127 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.750058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e08815-bed0-4bc5-8a9b-4dc041396127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.757178 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 769/1556] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:19.758131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef3300-d958-4660-b57c-dd1a3d670a75 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:19.764356 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:19.768527 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef3300-d958-4660-b57c-dd1a3d670a75 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:19.772845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-41e08815-bed0-4bc5-8a9b-4dc041396127 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:19.773635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b07c52-147c-46d4-85b8-96849d9fed93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:19.774241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-41e08815-bed0-4bc5-8a9b-4dc041396127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:19.774373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b07c52-147c-46d4-85b8-96849d9fed93 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.774522 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b07c52-147c-46d4-85b8-96849d9fed93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.774800 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b07c52-147c-46d4-85b8-96849d9fed93 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.775458 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b07c52-147c-46d4-85b8-96849d9fed93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:19.777422 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b07c52-147c-46d4-85b8-96849d9fed93 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:19.782338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 788/1557] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:19.783524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff53a7b-ad27-4863-be2c-d897ffe57367 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.783846 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff53a7b-ad27-4863-be2c-d897ffe57367 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.784129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff53a7b-ad27-4863-be2c-d897ffe57367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.804134 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 770/1558] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/users => generated 290 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:19.804694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0ff53a7b-ad27-4863-be2c-d897ffe57367 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:19.805769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c8f08-a81c-4c6b-b8a3-5e66e39e228b None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.805918 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0ff53a7b-ad27-4863-be2c-d897ffe57367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:19.806173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c8f08-a81c-4c6b-b8a3-5e66e39e228b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.806511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c8f08-a81c-4c6b-b8a3-5e66e39e228b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.814421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 789/1559] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:19.815609 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c6248-354a-45af-b7cd-abc8a9757156 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.815912 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c6248-354a-45af-b7cd-abc8a9757156 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.816183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c6248-354a-45af-b7cd-abc8a9757156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.826872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-8d6c8f08-a81c-4c6b-b8a3-5e66e39e228b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:19.828416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-8d6c8f08-a81c-4c6b-b8a3-5e66e39e228b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:19.835644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-106c6248-354a-45af-b7cd-abc8a9757156 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:19.836389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 771/1560] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:19.843857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-106c6248-354a-45af-b7cd-abc8a9757156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:19.843998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61854efe-2043-4e06-b1ed-3f21857549fc None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.843998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61854efe-2043-4e06-b1ed-3f21857549fc None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.843998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61854efe-2043-4e06-b1ed-3f21857549fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.845398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 790/1561] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:19.846266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c6248-354a-45af-b7cd-abc8a9757156 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:19.856551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c6248-354a-45af-b7cd-abc8a9757156 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:19.862462 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca237328-88b7-4307-b09f-a5c2c190c43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:19.863068 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca237328-88b7-4307-b09f-a5c2c190c43f admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.863344 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca237328-88b7-4307-b09f-a5c2c190c43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.863633 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca237328-88b7-4307-b09f-a5c2c190c43f admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.864303 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca237328-88b7-4307-b09f-a5c2c190c43f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:19.864605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca237328-88b7-4307-b09f-a5c2c190c43f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:19.866214 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca237328-88b7-4307-b09f-a5c2c190c43f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:19.866871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-61854efe-2043-4e06-b1ed-3f21857549fc None None] MFA Rules not processed for user `8c4727a0ef41431992676f231464f041`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:19.868117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-61854efe-2043-4e06-b1ed-3f21857549fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:19.884448 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 791/1562] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:19.885561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1d180-d25e-4045-b343-bd331af47982 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.885836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1d180-d25e-4045-b343-bd331af47982 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.886106 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1d180-d25e-4045-b343-bd331af47982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.906244 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-35f1d180-d25e-4045-b343-bd331af47982 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:19.907434 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-35f1d180-d25e-4045-b343-bd331af47982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:19.915424 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 792/1563] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:19.916409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9f636-67b0-4422-9527-5acb7b1da168 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.916685 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9f636-67b0-4422-9527-5acb7b1da168 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.916955 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9f636-67b0-4422-9527-5acb7b1da168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.935298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f5e9f636-67b0-4422-9527-5acb7b1da168 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:19.936540 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f5e9f636-67b0-4422-9527-5acb7b1da168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:19.942561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 772/1564] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:19.944680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 793/1565] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:19.944996 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61854efe-2043-4e06-b1ed-3f21857549fc None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:19.945315 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9f636-67b0-4422-9527-5acb7b1da168 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:19.959116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61854efe-2043-4e06-b1ed-3f21857549fc None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:19.959306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9f636-67b0-4422-9527-5acb7b1da168 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:19.964325 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac14c9-2459-42ca-ade2-fd9ece8f9604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:19.964847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368cfb6-5b30-489d-bb64-dbc74bc03124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:19.964964 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac14c9-2459-42ca-ade2-fd9ece8f9604 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.965191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac14c9-2459-42ca-ade2-fd9ece8f9604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.965456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368cfb6-5b30-489d-bb64-dbc74bc03124 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.965571 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac14c9-2459-42ca-ade2-fd9ece8f9604 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.965748 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368cfb6-5b30-489d-bb64-dbc74bc03124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.966012 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368cfb6-5b30-489d-bb64-dbc74bc03124 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.966133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac14c9-2459-42ca-ade2-fd9ece8f9604 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:19.966384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac14c9-2459-42ca-ade2-fd9ece8f9604 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:19.966655 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1368cfb6-5b30-489d-bb64-dbc74bc03124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:19.968677 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac14c9-2459-42ca-ade2-fd9ece8f9604 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:19.969253 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1368cfb6-5b30-489d-bb64-dbc74bc03124 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:19.980454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 773/1566] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:19.982106 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff1ace-c0a5-407c-b340-4be98825acf2 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:19.982400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff1ace-c0a5-407c-b340-4be98825acf2 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:19.982673 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff1ace-c0a5-407c-b340-4be98825acf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:19.986181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 794/1567] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:19.987111 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac14c9-2459-42ca-ade2-fd9ece8f9604 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.001587 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac14c9-2459-42ca-ade2-fd9ece8f9604 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.003318 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-7dff1ace-c0a5-407c-b340-4be98825acf2 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:20.004543 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7dff1ace-c0a5-407c-b340-4be98825acf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:20.006305 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f3014-730c-4bca-a6a6-5358ddadd353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.006904 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f3014-730c-4bca-a6a6-5358ddadd353 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.007190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f3014-730c-4bca-a6a6-5358ddadd353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.007466 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f3014-730c-4bca-a6a6-5358ddadd353 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.008144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f3014-730c-4bca-a6a6-5358ddadd353 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:20.008446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f3014-730c-4bca-a6a6-5358ddadd353 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.010047 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f3014-730c-4bca-a6a6-5358ddadd353 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.017316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 774/1568] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:20.017316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff1ace-c0a5-407c-b340-4be98825acf2 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.022659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff1ace-c0a5-407c-b340-4be98825acf2 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.030942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 795/1569] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:20.031106 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097ea6a-d5b8-4fbd-8c24-67b0191c8a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.031218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f3014-730c-4bca-a6a6-5358ddadd353 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.031300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097ea6a-d5b8-4fbd-8c24-67b0191c8a5c admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.031300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097ea6a-d5b8-4fbd-8c24-67b0191c8a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.031300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097ea6a-d5b8-4fbd-8c24-67b0191c8a5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.031300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097ea6a-d5b8-4fbd-8c24-67b0191c8a5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.031598 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097ea6a-d5b8-4fbd-8c24-67b0191c8a5c admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.039780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f3014-730c-4bca-a6a6-5358ddadd353 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.045908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cda3fe-4f19-4700-9f4c-092f3e4513de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.046513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cda3fe-4f19-4700-9f4c-092f3e4513de admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.046794 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cda3fe-4f19-4700-9f4c-092f3e4513de admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.047070 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cda3fe-4f19-4700-9f4c-092f3e4513de admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.047744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cda3fe-4f19-4700-9f4c-092f3e4513de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:20.048051 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cda3fe-4f19-4700-9f4c-092f3e4513de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.049666 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cda3fe-4f19-4700-9f4c-092f3e4513de admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.063070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 775/1570] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:20 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:20.063235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 796/1571] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:20.063235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cda3fe-4f19-4700-9f4c-092f3e4513de admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.064855 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55052244-efaa-4a96-9981-3d706d64e45c None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.065148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55052244-efaa-4a96-9981-3d706d64e45c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.065441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55052244-efaa-4a96-9981-3d706d64e45c None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.066696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 776/1572] 10.222.0.58 () {54 vars in 962 bytes} [Wed Aug 24 12:32:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:32:20.067396 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55052244-efaa-4a96-9981-3d706d64e45c None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.074052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cda3fe-4f19-4700-9f4c-092f3e4513de admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.122852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:20.133698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a17523-40e0-4c1f-aebf-5bcb23c9b4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.134315 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a17523-40e0-4c1f-aebf-5bcb23c9b4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.134607 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a17523-40e0-4c1f-aebf-5bcb23c9b4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.134879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a17523-40e0-4c1f-aebf-5bcb23c9b4e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.135527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a17523-40e0-4c1f-aebf-5bcb23c9b4e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:20.135861 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a17523-40e0-4c1f-aebf-5bcb23c9b4e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.137471 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a17523-40e0-4c1f-aebf-5bcb23c9b4e5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.138765 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55052244-efaa-4a96-9981-3d706d64e45c None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.144117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15de5f7-f334-4d4f-99e8-1d4d6b62fb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.144728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15de5f7-f334-4d4f-99e8-1d4d6b62fb57 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.145013 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15de5f7-f334-4d4f-99e8-1d4d6b62fb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.145286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15de5f7-f334-4d4f-99e8-1d4d6b62fb57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.145943 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15de5f7-f334-4d4f-99e8-1d4d6b62fb57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.147540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15de5f7-f334-4d4f-99e8-1d4d6b62fb57 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.149721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 797/1573] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:20.152494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a17523-40e0-4c1f-aebf-5bcb23c9b4e5 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.162476 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 777/1574] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:20.163403 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15de5f7-f334-4d4f-99e8-1d4d6b62fb57 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.167161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a17523-40e0-4c1f-aebf-5bcb23c9b4e5 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.172491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a761632-4d96-49fb-bfea-5d7086c44307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.173113 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a761632-4d96-49fb-bfea-5d7086c44307 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.173398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a761632-4d96-49fb-bfea-5d7086c44307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.173699 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a761632-4d96-49fb-bfea-5d7086c44307 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.174329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a761632-4d96-49fb-bfea-5d7086c44307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.176326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a761632-4d96-49fb-bfea-5d7086c44307 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.176932 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15de5f7-f334-4d4f-99e8-1d4d6b62fb57 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.181798 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7d1ab-5227-413b-bd06-152b27bd4281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.182399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7d1ab-5227-413b-bd06-152b27bd4281 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.182678 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7d1ab-5227-413b-bd06-152b27bd4281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.182973 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd7d1ab-5227-413b-bd06-152b27bd4281 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.183714 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7d1ab-5227-413b-bd06-152b27bd4281 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:20.184021 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7d1ab-5227-413b-bd06-152b27bd4281 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.185663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd7d1ab-5227-413b-bd06-152b27bd4281 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.198984 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 778/1575] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:20.199840 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7d1ab-5227-413b-bd06-152b27bd4281 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.209027 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 798/1576] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:20 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:20.210141 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a761632-4d96-49fb-bfea-5d7086c44307 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.215725 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd7d1ab-5227-413b-bd06-152b27bd4281 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.264034 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:20.276808 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a761632-4d96-49fb-bfea-5d7086c44307 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.276960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835c58b-5fc6-44bc-9f61-81c4efc9560b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.277497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835c58b-5fc6-44bc-9f61-81c4efc9560b admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.277841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835c58b-5fc6-44bc-9f61-81c4efc9560b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.278052 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835c58b-5fc6-44bc-9f61-81c4efc9560b admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.278677 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835c58b-5fc6-44bc-9f61-81c4efc9560b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.280728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835c58b-5fc6-44bc-9f61-81c4efc9560b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.281985 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7ef88-0f78-426f-ad19-15f2de29404a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.282583 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7ef88-0f78-426f-ad19-15f2de29404a admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.282864 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7ef88-0f78-426f-ad19-15f2de29404a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.283133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7ef88-0f78-426f-ad19-15f2de29404a admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.283785 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7ef88-0f78-426f-ad19-15f2de29404a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.284793 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-7eb7ef88-0f78-426f-ad19-15f2de29404a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:32:20.284793 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 24 12:32:20.284793 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: Aug 24 12:32:20.286121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7ef88-0f78-426f-ad19-15f2de29404a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.307663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 799/1577] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:20 2022] POST /identity/v3/users => generated 302 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:20.307977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 779/1578] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:20 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:20.312819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7ef88-0f78-426f-ad19-15f2de29404a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.313507 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd82a4-3931-462d-9efc-1ecbc83a5654 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.313800 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd82a4-3931-462d-9efc-1ecbc83a5654 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.314370 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd82a4-3931-462d-9efc-1ecbc83a5654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.322524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7ef88-0f78-426f-ad19-15f2de29404a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.352273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-1ebd82a4-3931-462d-9efc-1ecbc83a5654 None None] MFA Rules not processed for user `311dbed3ab0b44a7830a93227ff9ad33`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:20.353492 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1ebd82a4-3931-462d-9efc-1ecbc83a5654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:20.379448 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391b75f-9fd8-4e8f-a97d-fc0f12e41273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.380113 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391b75f-9fd8-4e8f-a97d-fc0f12e41273 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.380411 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391b75f-9fd8-4e8f-a97d-fc0f12e41273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.380780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391b75f-9fd8-4e8f-a97d-fc0f12e41273 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.381414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391b75f-9fd8-4e8f-a97d-fc0f12e41273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.383328 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391b75f-9fd8-4e8f-a97d-fc0f12e41273 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.412210 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 800/1579] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:20 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:20.413010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391b75f-9fd8-4e8f-a97d-fc0f12e41273 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.422659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 780/1580] 10.222.0.58 () {58 vars in 1059 bytes} [Wed Aug 24 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:20.423549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd82a4-3931-462d-9efc-1ecbc83a5654 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.433785 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd82a4-3931-462d-9efc-1ecbc83a5654 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.473600 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:20.479885 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391b75f-9fd8-4e8f-a97d-fc0f12e41273 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.485459 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbedf75-ad78-4df6-a8e8-8041df8bb59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.486066 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbedf75-ad78-4df6-a8e8-8041df8bb59f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.486341 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbedf75-ad78-4df6-a8e8-8041df8bb59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.486610 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbedf75-ad78-4df6-a8e8-8041df8bb59f admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.488299 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbedf75-ad78-4df6-a8e8-8041df8bb59f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.489297 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbedf75-ad78-4df6-a8e8-8041df8bb59f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.495837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967d699-7ba4-464b-8da3-33f79b10e2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.496500 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967d699-7ba4-464b-8da3-33f79b10e2ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.496782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967d699-7ba4-464b-8da3-33f79b10e2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.497055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d967d699-7ba4-464b-8da3-33f79b10e2ba admin admin] PATH_INFO: `/v3/projects/e35f67b84ccf4dcf9b239933a624c61d/users/9bf0e4a8956744a2aff4a3c2dea24b26/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.512753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d967d699-7ba4-464b-8da3-33f79b10e2ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e35f67b84ccf4dcf9b239933a624c61d, user_id=9bf0e4a8956744a2aff4a3c2dea24b26, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.513802 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: WARNING py.warnings [None req-d967d699-7ba4-464b-8da3-33f79b10e2ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:32:20.513802 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: warnings.warn(msg) Aug 24 12:32:20.513802 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: Aug 24 12:32:20.516366 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d967d699-7ba4-464b-8da3-33f79b10e2ba admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.517189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 801/1581] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:20 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:20.517989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbedf75-ad78-4df6-a8e8-8041df8bb59f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.529798 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 781/1582] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:20 2022] PUT /identity/v3/projects/e35f67b84ccf4dcf9b239933a624c61d/users/9bf0e4a8956744a2aff4a3c2dea24b26/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:20.531206 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967d699-7ba4-464b-8da3-33f79b10e2ba admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.576010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:20.588207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbedf75-ad78-4df6-a8e8-8041df8bb59f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.593366 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7775aaa-588b-4908-951c-190e199ed8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.593971 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7775aaa-588b-4908-951c-190e199ed8cb admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.594251 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7775aaa-588b-4908-951c-190e199ed8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.594522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7775aaa-588b-4908-951c-190e199ed8cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.595142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7775aaa-588b-4908-951c-190e199ed8cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.597092 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7775aaa-588b-4908-951c-190e199ed8cb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.597808 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d967d699-7ba4-464b-8da3-33f79b10e2ba admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.602451 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a792f-4539-4e4c-b52c-f022afff5be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.603063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a792f-4539-4e4c-b52c-f022afff5be2 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.603347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a792f-4539-4e4c-b52c-f022afff5be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.603633 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a792f-4539-4e4c-b52c-f022afff5be2 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.604273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a792f-4539-4e4c-b52c-f022afff5be2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.606184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a792f-4539-4e4c-b52c-f022afff5be2 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.626276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 802/1583] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:20 2022] POST /identity/v3/projects => generated 393 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:20.627105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7775aaa-588b-4908-951c-190e199ed8cb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.635339 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 782/1584] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:20 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:20.635339 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a792f-4539-4e4c-b52c-f022afff5be2 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.643997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a792f-4539-4e4c-b52c-f022afff5be2 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.698760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7775aaa-588b-4908-951c-190e199ed8cb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.700493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fa5be-9fc1-4b1a-af30-92ab16fbb533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.701120 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fa5be-9fc1-4b1a-af30-92ab16fbb533 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.701388 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fa5be-9fc1-4b1a-af30-92ab16fbb533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.701665 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6fa5be-9fc1-4b1a-af30-92ab16fbb533 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.702789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fa5be-9fc1-4b1a-af30-92ab16fbb533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.703733 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee981ecd-fcaf-4b7d-b4e6-3d9423c00593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.704336 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee981ecd-fcaf-4b7d-b4e6-3d9423c00593 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.704622 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee981ecd-fcaf-4b7d-b4e6-3d9423c00593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.704789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6fa5be-9fc1-4b1a-af30-92ab16fbb533 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.704893 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee981ecd-fcaf-4b7d-b4e6-3d9423c00593 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.705525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee981ecd-fcaf-4b7d-b4e6-3d9423c00593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.707143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee981ecd-fcaf-4b7d-b4e6-3d9423c00593 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.717274 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 803/1585] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:20.718179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee981ecd-fcaf-4b7d-b4e6-3d9423c00593 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.728233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee981ecd-fcaf-4b7d-b4e6-3d9423c00593 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.728486 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 783/1586] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:20 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:20.729333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fa5be-9fc1-4b1a-af30-92ab16fbb533 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.737519 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e955e6-d004-4a14-9962-dfafdeda7617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.738125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e955e6-d004-4a14-9962-dfafdeda7617 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.738408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e955e6-d004-4a14-9962-dfafdeda7617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.738681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e955e6-d004-4a14-9962-dfafdeda7617 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.739304 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e955e6-d004-4a14-9962-dfafdeda7617 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.741272 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e955e6-d004-4a14-9962-dfafdeda7617 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.767545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 804/1587] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:20 2022] POST /identity/v3/users => generated 293 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:20.767545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e955e6-d004-4a14-9962-dfafdeda7617 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.774633 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e955e6-d004-4a14-9962-dfafdeda7617 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.777278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:20.779569 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba01836d-f514-488f-83a8-bb99c90938ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.780243 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01836d-f514-488f-83a8-bb99c90938ae admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.780528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01836d-f514-488f-83a8-bb99c90938ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.780801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba01836d-f514-488f-83a8-bb99c90938ae admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.781448 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01836d-f514-488f-83a8-bb99c90938ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.783491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba01836d-f514-488f-83a8-bb99c90938ae admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.805087 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6fa5be-9fc1-4b1a-af30-92ab16fbb533 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.805524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 805/1588] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:20 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:20.806747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1adb7-db2f-40e4-9b82-1d20a9e9d1b7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.807036 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1adb7-db2f-40e4-9b82-1d20a9e9d1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.807310 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1adb7-db2f-40e4-9b82-1d20a9e9d1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.814950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5af04-4062-4cad-b4f2-bc71d3cbc50d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.815576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5af04-4062-4cad-b4f2-bc71d3cbc50d service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.815896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5af04-4062-4cad-b4f2-bc71d3cbc50d service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.816173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a5af04-4062-4cad-b4f2-bc71d3cbc50d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.837837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-40e1adb7-db2f-40e4-9b82-1d20a9e9d1b7 None None] MFA Rules not processed for user `9bf0e4a8956744a2aff4a3c2dea24b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:20.839148 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-40e1adb7-db2f-40e4-9b82-1d20a9e9d1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:20.881602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5af04-4062-4cad-b4f2-bc71d3cbc50d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.883220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a5af04-4062-4cad-b4f2-bc71d3cbc50d service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.893184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 784/1589] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:20.894079 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5af04-4062-4cad-b4f2-bc71d3cbc50d service nova] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.903841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a5af04-4062-4cad-b4f2-bc71d3cbc50d service nova] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.908757 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59df09-2003-4fc1-8764-4ac3ef8ca94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.909573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59df09-2003-4fc1-8764-4ac3ef8ca94e admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.909716 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59df09-2003-4fc1-8764-4ac3ef8ca94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.909876 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe59df09-2003-4fc1-8764-4ac3ef8ca94e admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.910480 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59df09-2003-4fc1-8764-4ac3ef8ca94e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.912419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe59df09-2003-4fc1-8764-4ac3ef8ca94e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.915694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 806/1590] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:20.917157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e1adb7-db2f-40e4-9b82-1d20a9e9d1b7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.930527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e1adb7-db2f-40e4-9b82-1d20a9e9d1b7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.939115 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 785/1591] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:20 2022] POST /identity/v3/users => generated 297 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:20.939285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0323989-6c19-43eb-aa46-6d5d3e1b32c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.939285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0323989-6c19-43eb-aa46-6d5d3e1b32c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.939285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0323989-6c19-43eb-aa46-6d5d3e1b32c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.939285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0323989-6c19-43eb-aa46-6d5d3e1b32c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.939962 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59df09-2003-4fc1-8764-4ac3ef8ca94e admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.940066 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0323989-6c19-43eb-aa46-6d5d3e1b32c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.940428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0323989-6c19-43eb-aa46-6d5d3e1b32c9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.949838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe59df09-2003-4fc1-8764-4ac3ef8ca94e admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.957007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 807/1592] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:20.957007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0323989-6c19-43eb-aa46-6d5d3e1b32c9 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.958824 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b6d2b-87f2-4575-99b1-c5d0d76941e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.959549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b6d2b-87f2-4575-99b1-c5d0d76941e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.959945 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b6d2b-87f2-4575-99b1-c5d0d76941e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.960313 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b6d2b-87f2-4575-99b1-c5d0d76941e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.961050 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b6d2b-87f2-4575-99b1-c5d0d76941e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.962780 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b6d2b-87f2-4575-99b1-c5d0d76941e6 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.965774 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0323989-6c19-43eb-aa46-6d5d3e1b32c9 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.971287 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6ed21-3265-4411-ab4b-ef1ffdc385c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.971990 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6ed21-3265-4411-ab4b-ef1ffdc385c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.972355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6ed21-3265-4411-ab4b-ef1ffdc385c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.972702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f6ed21-3265-4411-ab4b-ef1ffdc385c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.973453 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6ed21-3265-4411-ab4b-ef1ffdc385c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.974389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 786/1593] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:20.975132 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b6d2b-87f2-4575-99b1-c5d0d76941e6 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.978557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f6ed21-3265-4411-ab4b-ef1ffdc385c4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:20.985316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b6d2b-87f2-4575-99b1-c5d0d76941e6 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:20.990049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de598427-a18d-4137-92b0-31f706c2be30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:20.993497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 808/1594] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:20.993971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de598427-a18d-4137-92b0-31f706c2be30 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:20.993971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de598427-a18d-4137-92b0-31f706c2be30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:20.993971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de598427-a18d-4137-92b0-31f706c2be30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:20.994236 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6ed21-3265-4411-ab4b-ef1ffdc385c4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:20.994328 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de598427-a18d-4137-92b0-31f706c2be30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:20.994328 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de598427-a18d-4137-92b0-31f706c2be30 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.005491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f6ed21-3265-4411-ab4b-ef1ffdc385c4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.009117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 787/1595] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:21.012381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de598427-a18d-4137-92b0-31f706c2be30 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.014210 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f504ac-cb87-4394-9b6c-85c351e88c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.015299 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f504ac-cb87-4394-9b6c-85c351e88c6c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.015715 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f504ac-cb87-4394-9b6c-85c351e88c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.016121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f504ac-cb87-4394-9b6c-85c351e88c6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.016924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f504ac-cb87-4394-9b6c-85c351e88c6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.018755 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f504ac-cb87-4394-9b6c-85c351e88c6c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.020699 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:21.021750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de598427-a18d-4137-92b0-31f706c2be30 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.028164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62e18d-0e91-42b4-8739-65f2a39ab423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.029928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62e18d-0e91-42b4-8739-65f2a39ab423 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.030923 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62e18d-0e91-42b4-8739-65f2a39ab423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.031367 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62e18d-0e91-42b4-8739-65f2a39ab423 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.032232 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62e18d-0e91-42b4-8739-65f2a39ab423 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.035026 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62e18d-0e91-42b4-8739-65f2a39ab423 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.036473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 809/1596] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:21.037360 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f504ac-cb87-4394-9b6c-85c351e88c6c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.048782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f504ac-cb87-4394-9b6c-85c351e88c6c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.053631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f9361-fe67-41c2-8b10-0ce184ccb6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.054224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f9361-fe67-41c2-8b10-0ce184ccb6c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.054505 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f9361-fe67-41c2-8b10-0ce184ccb6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.054783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f9361-fe67-41c2-8b10-0ce184ccb6c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.055417 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f9361-fe67-41c2-8b10-0ce184ccb6c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.057059 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f9361-fe67-41c2-8b10-0ce184ccb6c3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.070969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 810/1597] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:21.070969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f9361-fe67-41c2-8b10-0ce184ccb6c3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.079321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 788/1598] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:21 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:21.080693 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62e18d-0e91-42b4-8739-65f2a39ab423 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.082220 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f9361-fe67-41c2-8b10-0ce184ccb6c3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.144051 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eae3d-681b-4180-9721-ee1fe6a9c747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.144666 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eae3d-681b-4180-9721-ee1fe6a9c747 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.144952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eae3d-681b-4180-9721-ee1fe6a9c747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.145228 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769eae3d-681b-4180-9721-ee1fe6a9c747 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.145875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eae3d-681b-4180-9721-ee1fe6a9c747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.147508 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769eae3d-681b-4180-9721-ee1fe6a9c747 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.154407 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62e18d-0e91-42b4-8739-65f2a39ab423 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.157633 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 811/1599] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:21.159307 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df819a67-9a54-4c1a-a40a-e7e77ff7edef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.159998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df819a67-9a54-4c1a-a40a-e7e77ff7edef admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.160283 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df819a67-9a54-4c1a-a40a-e7e77ff7edef admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.160577 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df819a67-9a54-4c1a-a40a-e7e77ff7edef admin admin] PATH_INFO: `/v3/projects/3a1d2aaa6b3a44a9a9e7daabd3c7a028/users/b60bc0c27b494f83a3b342ed407cb6e9/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.160793 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eae3d-681b-4180-9721-ee1fe6a9c747 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.170291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769eae3d-681b-4180-9721-ee1fe6a9c747 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.175207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b4e45-4184-4136-a55e-4f4d6fa2a7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.175749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df819a67-9a54-4c1a-a40a-e7e77ff7edef admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a1d2aaa6b3a44a9a9e7daabd3c7a028, user_id=b60bc0c27b494f83a3b342ed407cb6e9, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.175982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b4e45-4184-4136-a55e-4f4d6fa2a7ed admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.176390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b4e45-4184-4136-a55e-4f4d6fa2a7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.176672 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13b4e45-4184-4136-a55e-4f4d6fa2a7ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.177309 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b4e45-4184-4136-a55e-4f4d6fa2a7ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.178456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df819a67-9a54-4c1a-a40a-e7e77ff7edef admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.178986 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13b4e45-4184-4136-a55e-4f4d6fa2a7ed admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.192946 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:21.197499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 812/1600] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:21.198429 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b4e45-4184-4136-a55e-4f4d6fa2a7ed admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.198728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 789/1601] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:21 2022] PUT /identity/v3/projects/3a1d2aaa6b3a44a9a9e7daabd3c7a028/users/b60bc0c27b494f83a3b342ed407cb6e9/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:21.200084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df819a67-9a54-4c1a-a40a-e7e77ff7edef admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.216257 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13b4e45-4184-4136-a55e-4f4d6fa2a7ed admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.268956 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df819a67-9a54-4c1a-a40a-e7e77ff7edef admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.273440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb436b-3a58-4f1e-85c1-f3f30a0e5d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.274050 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb436b-3a58-4f1e-85c1-f3f30a0e5d95 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.274333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb436b-3a58-4f1e-85c1-f3f30a0e5d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.274609 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb436b-3a58-4f1e-85c1-f3f30a0e5d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.275122 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f6d20-f1db-45e7-8854-fc44800d095d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.275243 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb436b-3a58-4f1e-85c1-f3f30a0e5d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.275781 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f6d20-f1db-45e7-8854-fc44800d095d admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.276064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f6d20-f1db-45e7-8854-fc44800d095d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.276336 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f6d20-f1db-45e7-8854-fc44800d095d admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.276886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb436b-3a58-4f1e-85c1-f3f30a0e5d95 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.276992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f6d20-f1db-45e7-8854-fc44800d095d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.278818 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f6d20-f1db-45e7-8854-fc44800d095d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.287687 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 790/1602] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:21.289292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb436b-3a58-4f1e-85c1-f3f30a0e5d95 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.289764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 813/1603] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:21.290622 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f6d20-f1db-45e7-8854-fc44800d095d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.301228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb436b-3a58-4f1e-85c1-f3f30a0e5d95 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.304032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f6d20-f1db-45e7-8854-fc44800d095d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.307250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85641b2d-0609-4e26-b487-e7f12336c336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.307891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85641b2d-0609-4e26-b487-e7f12336c336 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.308197 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85641b2d-0609-4e26-b487-e7f12336c336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.308446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85641b2d-0609-4e26-b487-e7f12336c336 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.308653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e1563-a4f2-470c-985b-5771e9673e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.309091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85641b2d-0609-4e26-b487-e7f12336c336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.309284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e1563-a4f2-470c-985b-5771e9673e8d admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.309569 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e1563-a4f2-470c-985b-5771e9673e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.309842 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e1563-a4f2-470c-985b-5771e9673e8d admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.310471 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e1563-a4f2-470c-985b-5771e9673e8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.310687 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85641b2d-0609-4e26-b487-e7f12336c336 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.312495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e1563-a4f2-470c-985b-5771e9673e8d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.320908 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 791/1604] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:21.322116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374b337-2434-4fe4-941b-8297be10b8ef None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.322418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374b337-2434-4fe4-941b-8297be10b8ef None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.322676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374b337-2434-4fe4-941b-8297be10b8ef None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.323918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 792/1605] 10.222.0.58 () {54 vars in 912 bytes} [Wed Aug 24 12:32:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:32:21.328565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374b337-2434-4fe4-941b-8297be10b8ef None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.337733 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 814/1606] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:21 2022] POST /identity/v3/users => generated 290 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:21.338869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e1563-a4f2-470c-985b-5771e9673e8d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.341421 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374b337-2434-4fe4-941b-8297be10b8ef None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.346845 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7692f-f2e8-486f-a6e5-63789b7aad81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.347484 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7692f-f2e8-486f-a6e5-63789b7aad81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.347828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7692f-f2e8-486f-a6e5-63789b7aad81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.348103 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7692f-f2e8-486f-a6e5-63789b7aad81 admin admin] PATH_INFO: `/v3/projects/36bd9970f0694314b966d86f13b83d4b/users/ed0d97f3f08f46af851492aee6a8d0ef/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.349140 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e1563-a4f2-470c-985b-5771e9673e8d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.353901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cef278-b92c-4215-ae92-7fa0fbb407cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.354539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cef278-b92c-4215-ae92-7fa0fbb407cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.354820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cef278-b92c-4215-ae92-7fa0fbb407cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.355090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cef278-b92c-4215-ae92-7fa0fbb407cf admin admin] PATH_INFO: `/v3/projects/b38fa406b760405f8d3004db272f1e52/users/28dac3ee2d5c4ff5ba2d15b59e57d2a7/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.362718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7692f-f2e8-486f-a6e5-63789b7aad81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36bd9970f0694314b966d86f13b83d4b, user_id=ed0d97f3f08f46af851492aee6a8d0ef, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.365400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7692f-f2e8-486f-a6e5-63789b7aad81 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.370105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cef278-b92c-4215-ae92-7fa0fbb407cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38fa406b760405f8d3004db272f1e52, user_id=28dac3ee2d5c4ff5ba2d15b59e57d2a7, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.372869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cef278-b92c-4215-ae92-7fa0fbb407cf admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.379924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 793/1607] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:21 2022] PUT /identity/v3/projects/36bd9970f0694314b966d86f13b83d4b/users/ed0d97f3f08f46af851492aee6a8d0ef/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:21.381116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eaa86b-3eee-42fe-bc63-c9da90be2e8d None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.381399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eaa86b-3eee-42fe-bc63-c9da90be2e8d None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.381673 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eaa86b-3eee-42fe-bc63-c9da90be2e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.389034 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 815/1608] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:21 2022] PUT /identity/v3/projects/b38fa406b760405f8d3004db272f1e52/users/28dac3ee2d5c4ff5ba2d15b59e57d2a7/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:21.391207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cef278-b92c-4215-ae92-7fa0fbb407cf admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.412130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-30eaa86b-3eee-42fe-bc63-c9da90be2e8d None None] MFA Rules not processed for user `b60bc0c27b494f83a3b342ed407cb6e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:21.413355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-30eaa86b-3eee-42fe-bc63-c9da90be2e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:21.419399 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:21.455924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cef278-b92c-4215-ae92-7fa0fbb407cf admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.460554 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5aae6-f5fe-4c87-b4ba-40edd46a3dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.461223 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5aae6-f5fe-4c87-b4ba-40edd46a3dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.461504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5aae6-f5fe-4c87-b4ba-40edd46a3dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.461779 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5aae6-f5fe-4c87-b4ba-40edd46a3dc3 admin admin] PATH_INFO: `/v3/projects/2fd2c804427749c68ff038d019a1628b/users/539573490e0d4e128caf93c51567ed79/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.477252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5aae6-f5fe-4c87-b4ba-40edd46a3dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd2c804427749c68ff038d019a1628b, user_id=539573490e0d4e128caf93c51567ed79, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.478899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 794/1609] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:21 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:21.479851 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eaa86b-3eee-42fe-bc63-c9da90be2e8d None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.479967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5aae6-f5fe-4c87-b4ba-40edd46a3dc3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.490741 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eaa86b-3eee-42fe-bc63-c9da90be2e8d None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.492933 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 816/1610] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:21 2022] PUT /identity/v3/projects/2fd2c804427749c68ff038d019a1628b/users/539573490e0d4e128caf93c51567ed79/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:21.493779 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5aae6-f5fe-4c87-b4ba-40edd46a3dc3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.541560 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:21.552257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23159f0b-2215-42e8-b47d-7fcdecd55e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.553451 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23159f0b-2215-42e8-b47d-7fcdecd55e88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.553736 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23159f0b-2215-42e8-b47d-7fcdecd55e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.554011 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23159f0b-2215-42e8-b47d-7fcdecd55e88 admin admin] PATH_INFO: `/v3/projects/f3a2fec602684ab1a454b8da589b4787/users/9aa753d1e2c4430498d5b28f0304952f/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.556449 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5aae6-f5fe-4c87-b4ba-40edd46a3dc3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.561136 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1ebc2-ffe2-49b1-aea1-972a2cbccfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.562289 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1ebc2-ffe2-49b1-aea1-972a2cbccfd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.562576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1ebc2-ffe2-49b1-aea1-972a2cbccfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.562856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1ebc2-ffe2-49b1-aea1-972a2cbccfd4 admin admin] PATH_INFO: `/v3/projects/9afa0216c9e140bb82f56f14ccfb7bb6/users/9138ac21bbdb4577a7f43b5ab471d572/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.568906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23159f0b-2215-42e8-b47d-7fcdecd55e88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3a2fec602684ab1a454b8da589b4787, user_id=9aa753d1e2c4430498d5b28f0304952f, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.571585 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23159f0b-2215-42e8-b47d-7fcdecd55e88 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.579054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f1ebc2-ffe2-49b1-aea1-972a2cbccfd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9afa0216c9e140bb82f56f14ccfb7bb6, user_id=9138ac21bbdb4577a7f43b5ab471d572, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.581866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f1ebc2-ffe2-49b1-aea1-972a2cbccfd4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.585483 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 795/1611] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:21 2022] PUT /identity/v3/projects/f3a2fec602684ab1a454b8da589b4787/users/9aa753d1e2c4430498d5b28f0304952f/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:21.586933 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3365e8-c2f8-4545-aee8-2a28125fdc17 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.587296 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3365e8-c2f8-4545-aee8-2a28125fdc17 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.587616 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3365e8-c2f8-4545-aee8-2a28125fdc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.597700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 817/1612] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:21 2022] PUT /identity/v3/projects/9afa0216c9e140bb82f56f14ccfb7bb6/users/9138ac21bbdb4577a7f43b5ab471d572/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:21.599031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1ebc2-ffe2-49b1-aea1-972a2cbccfd4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.608198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-fb3365e8-c2f8-4545-aee8-2a28125fdc17 None None] MFA Rules not processed for user `311dbed3ab0b44a7830a93227ff9ad33`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:21.609508 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fb3365e8-c2f8-4545-aee8-2a28125fdc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:21.667436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1ebc2-ffe2-49b1-aea1-972a2cbccfd4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.672532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711adaa1-a417-4ccc-88b5-ce181708e1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.673154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711adaa1-a417-4ccc-88b5-ce181708e1ac admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.673431 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711adaa1-a417-4ccc-88b5-ce181708e1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.673709 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711adaa1-a417-4ccc-88b5-ce181708e1ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.674386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711adaa1-a417-4ccc-88b5-ce181708e1ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.676080 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711adaa1-a417-4ccc-88b5-ce181708e1ac admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.678037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 796/1613] 10.222.0.58 () {58 vars in 1009 bytes} [Wed Aug 24 12:32:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:21.679527 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79c4ac-bac1-4570-b5b9-1c53fa6423f1 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.679884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79c4ac-bac1-4570-b5b9-1c53fa6423f1 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.680161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79c4ac-bac1-4570-b5b9-1c53fa6423f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.687476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 818/1614] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:21.688772 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049a67c-6723-4b05-ac1d-9465edaf8d88 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.689062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049a67c-6723-4b05-ac1d-9465edaf8d88 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.689476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049a67c-6723-4b05-ac1d-9465edaf8d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.707464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-cc79c4ac-bac1-4570-b5b9-1c53fa6423f1 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:21.708703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-cc79c4ac-bac1-4570-b5b9-1c53fa6423f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:21.710631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1049a67c-6723-4b05-ac1d-9465edaf8d88 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:21.712003 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1049a67c-6723-4b05-ac1d-9465edaf8d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:21.716523 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 797/1615] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:21.717278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79c4ac-bac1-4570-b5b9-1c53fa6423f1 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.722595 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 819/1616] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:21.723694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83617e0-48e1-456f-b792-c45e56156388 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.723984 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83617e0-48e1-456f-b792-c45e56156388 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.724255 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83617e0-48e1-456f-b792-c45e56156388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.726984 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79c4ac-bac1-4570-b5b9-1c53fa6423f1 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.732234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d5235-647c-4fd0-9530-db8f205f42c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.732844 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d5235-647c-4fd0-9530-db8f205f42c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.733127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d5235-647c-4fd0-9530-db8f205f42c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.733739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d5235-647c-4fd0-9530-db8f205f42c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.734381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d5235-647c-4fd0-9530-db8f205f42c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.736375 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d5235-647c-4fd0-9530-db8f205f42c1 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.744344 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d83617e0-48e1-456f-b792-c45e56156388 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:21.745548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d83617e0-48e1-456f-b792-c45e56156388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:21.753700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 820/1617] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:21.754884 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e204df-ddd3-401f-9100-d5c84b68f7b8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.755166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e204df-ddd3-401f-9100-d5c84b68f7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.755437 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e204df-ddd3-401f-9100-d5c84b68f7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.771442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 798/1618] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:21 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:21.773020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edd00a-6c8a-4811-a319-4aa31babe984 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.773309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edd00a-6c8a-4811-a319-4aa31babe984 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.773585 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edd00a-6c8a-4811-a319-4aa31babe984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.778821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c1e204df-ddd3-401f-9100-d5c84b68f7b8 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:21.780066 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c1e204df-ddd3-401f-9100-d5c84b68f7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:21.788680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:21.792996 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-13edd00a-6c8a-4811-a319-4aa31babe984 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:21.794159 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-13edd00a-6c8a-4811-a319-4aa31babe984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:21.845415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 821/1619] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:21.845415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e204df-ddd3-401f-9100-d5c84b68f7b8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.854167 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 799/1620] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:21.855304 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13edd00a-6c8a-4811-a319-4aa31babe984 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.856806 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e204df-ddd3-401f-9100-d5c84b68f7b8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.862724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efb882-286a-476c-bbd4-fb3c99fd29e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.863335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efb882-286a-476c-bbd4-fb3c99fd29e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.863643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efb882-286a-476c-bbd4-fb3c99fd29e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.863919 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efb882-286a-476c-bbd4-fb3c99fd29e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.864590 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4efb882-286a-476c-bbd4-fb3c99fd29e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.866176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4efb882-286a-476c-bbd4-fb3c99fd29e3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.877852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 822/1621] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:21.878707 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efb882-286a-476c-bbd4-fb3c99fd29e3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.899596 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:21.915229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13edd00a-6c8a-4811-a319-4aa31babe984 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.919056 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cb2b2-7811-471c-a44a-55860912add5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.919693 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cb2b2-7811-471c-a44a-55860912add5 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.919985 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cb2b2-7811-471c-a44a-55860912add5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.920259 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cb2b2-7811-471c-a44a-55860912add5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.937646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efb882-286a-476c-bbd4-fb3c99fd29e3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.941420 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1224a83-ec2a-4a45-a518-866fd4cf79b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.942018 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1224a83-ec2a-4a45-a518-866fd4cf79b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.942298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1224a83-ec2a-4a45-a518-866fd4cf79b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.942575 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1224a83-ec2a-4a45-a518-866fd4cf79b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.953143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1224a83-ec2a-4a45-a518-866fd4cf79b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.954562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1224a83-ec2a-4a45-a518-866fd4cf79b0 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.964198 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 823/1622] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:21.964790 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1224a83-ec2a-4a45-a518-866fd4cf79b0 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:21.973182 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1224a83-ec2a-4a45-a518-866fd4cf79b0 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:21.977113 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb049a21-ed45-433b-b3a3-52723e72ace8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:21.977713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049a21-ed45-433b-b3a3-52723e72ace8 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:21.977995 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049a21-ed45-433b-b3a3-52723e72ace8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:21.978274 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049a21-ed45-433b-b3a3-52723e72ace8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:21.989134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb049a21-ed45-433b-b3a3-52723e72ace8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.990691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb049a21-ed45-433b-b3a3-52723e72ace8 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:21.992658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461cb2b2-7811-471c-a44a-55860912add5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:21.994380 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461cb2b2-7811-471c-a44a-55860912add5 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:22.001182 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 824/1623] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:22.005236 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb049a21-ed45-433b-b3a3-52723e72ace8 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:22.013398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 800/1624] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:22.015340 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cb2b2-7811-471c-a44a-55860912add5 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:22.025565 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb049a21-ed45-433b-b3a3-52723e72ace8 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:22.031292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2dab0-0302-4288-b487-0812693fc0f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:22.032011 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cb2b2-7811-471c-a44a-55860912add5 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:22.032144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2dab0-0302-4288-b487-0812693fc0f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:22.032335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2dab0-0302-4288-b487-0812693fc0f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:22.037582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ad350-758c-4b3f-8ed4-82b042301a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:22.038384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ad350-758c-4b3f-8ed4-82b042301a48 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:22.038668 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ad350-758c-4b3f-8ed4-82b042301a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:22.038940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138ad350-758c-4b3f-8ed4-82b042301a48 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:22.039357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2dab0-0302-4288-b487-0812693fc0f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:22.039616 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ad350-758c-4b3f-8ed4-82b042301a48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:22.042274 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138ad350-758c-4b3f-8ed4-82b042301a48 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:22.061248 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2dab0-0302-4288-b487-0812693fc0f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:22.063397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2dab0-0302-4288-b487-0812693fc0f7 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:22.073738 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 801/1625] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:22 2022] POST /identity/v3/users => generated 309 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:22.075011 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ad350-758c-4b3f-8ed4-82b042301a48 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:22.079691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 825/1626] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:22.080661 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2dab0-0302-4288-b487-0812693fc0f7 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:22.094409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2dab0-0302-4288-b487-0812693fc0f7 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:22.103788 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78935598-0a8e-42d1-879c-c8d435e7d61c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:22.106335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78935598-0a8e-42d1-879c-c8d435e7d61c service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:22.106613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78935598-0a8e-42d1-879c-c8d435e7d61c service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:22.107184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78935598-0a8e-42d1-879c-c8d435e7d61c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:22.109378 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138ad350-758c-4b3f-8ed4-82b042301a48 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:22.118194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858039e4-b7ea-4752-926b-91d69d021227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:22.120283 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858039e4-b7ea-4752-926b-91d69d021227 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:22.120624 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858039e4-b7ea-4752-926b-91d69d021227 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:22.120841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858039e4-b7ea-4752-926b-91d69d021227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:22.128942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78935598-0a8e-42d1-879c-c8d435e7d61c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:22.131500 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78935598-0a8e-42d1-879c-c8d435e7d61c service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:22.142816 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858039e4-b7ea-4752-926b-91d69d021227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:22.144682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858039e4-b7ea-4752-926b-91d69d021227 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:22.158664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 826/1627] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:22.159762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78935598-0a8e-42d1-879c-c8d435e7d61c service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:22.168878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 802/1628] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:22.169907 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858039e4-b7ea-4752-926b-91d69d021227 service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:22.172786 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78935598-0a8e-42d1-879c-c8d435e7d61c service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:22.179029 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c42544f-16f8-43a8-9732-c2a16d0b1dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:22.180124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42544f-16f8-43a8-9732-c2a16d0b1dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:22.180624 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42544f-16f8-43a8-9732-c2a16d0b1dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:22.181111 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c42544f-16f8-43a8-9732-c2a16d0b1dfe admin admin] PATH_INFO: `/v3/projects/fa767e5b2ae34a91943e9449dff5476f/users/2061826a8a614ddeb070c31f54211fd9/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:22.195435 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858039e4-b7ea-4752-926b-91d69d021227 service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:22.201151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42544f-16f8-43a8-9732-c2a16d0b1dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa767e5b2ae34a91943e9449dff5476f, user_id=2061826a8a614ddeb070c31f54211fd9, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:22.205207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c42544f-16f8-43a8-9732-c2a16d0b1dfe admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:22.207988 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825dec0-eb1d-4612-a64f-62b9a1719139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:22.209595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825dec0-eb1d-4612-a64f-62b9a1719139 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:22.209874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825dec0-eb1d-4612-a64f-62b9a1719139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:22.210143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825dec0-eb1d-4612-a64f-62b9a1719139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:22.210808 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825dec0-eb1d-4612-a64f-62b9a1719139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:22.213898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825dec0-eb1d-4612-a64f-62b9a1719139 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:22.237230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 803/1629] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:22.241498 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825dec0-eb1d-4612-a64f-62b9a1719139 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:22.245709 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 827/1630] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:22 2022] PUT /identity/v3/projects/fa767e5b2ae34a91943e9449dff5476f/users/2061826a8a614ddeb070c31f54211fd9/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:22.257230 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6be4d8-2d0f-41df-976a-297561af14e6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:22.257742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6be4d8-2d0f-41df-976a-297561af14e6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:22.258175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6be4d8-2d0f-41df-976a-297561af14e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:22.271433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825dec0-eb1d-4612-a64f-62b9a1719139 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:22.344210 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5d6be4d8-2d0f-41df-976a-297561af14e6 None None] MFA Rules not processed for user `2061826a8a614ddeb070c31f54211fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:22.345580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5d6be4d8-2d0f-41df-976a-297561af14e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:22.366655 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c965411-b4a8-4c6c-8673-b0509630fbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:22.367601 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c965411-b4a8-4c6c-8673-b0509630fbec admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:22.368075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c965411-b4a8-4c6c-8673-b0509630fbec admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:22.368499 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c965411-b4a8-4c6c-8673-b0509630fbec admin admin] PATH_INFO: `/v3/projects/803aa3005d4b46f9bf0d7366ac575f96/users/edfade728cbd4bc8837740dddcb5e440/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:22.384125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:22.408770 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c965411-b4a8-4c6c-8673-b0509630fbec admin admin] RBAC: Authorizing `identity:create_grant(project_id=803aa3005d4b46f9bf0d7366ac575f96, user_id=edfade728cbd4bc8837740dddcb5e440, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:22.413907 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c965411-b4a8-4c6c-8673-b0509630fbec admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:22.449462 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 804/1631] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:22 2022] PUT /identity/v3/projects/803aa3005d4b46f9bf0d7366ac575f96/users/edfade728cbd4bc8837740dddcb5e440/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:22.449788 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 828/1632] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:22.450926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c965411-b4a8-4c6c-8673-b0509630fbec admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:22.453966 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6be4d8-2d0f-41df-976a-297561af14e6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:22.476640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6be4d8-2d0f-41df-976a-297561af14e6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:22.496930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:22.621209 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61afe6d-133e-42f4-bc45-5e4835e8e6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:22.622480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61afe6d-133e-42f4-bc45-5e4835e8e6b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:22.625865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61afe6d-133e-42f4-bc45-5e4835e8e6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:22.626172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61afe6d-133e-42f4-bc45-5e4835e8e6b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:22.629049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61afe6d-133e-42f4-bc45-5e4835e8e6b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:22.634414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c965411-b4a8-4c6c-8673-b0509630fbec admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:22.636593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61afe6d-133e-42f4-bc45-5e4835e8e6b9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:22.646974 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4b3a0-77eb-45bd-9a5b-bfb85939f375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:22.648588 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4b3a0-77eb-45bd-9a5b-bfb85939f375 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:22.648886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4b3a0-77eb-45bd-9a5b-bfb85939f375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:22.650296 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4b3a0-77eb-45bd-9a5b-bfb85939f375 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:22.651186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4b3a0-77eb-45bd-9a5b-bfb85939f375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:22.658670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4b3a0-77eb-45bd-9a5b-bfb85939f375 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:22.683700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 805/1633] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:22.687583 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4b3a0-77eb-45bd-9a5b-bfb85939f375 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:22.701389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 829/1634] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:22 2022] POST /identity/v3/projects => generated 379 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:22.706055 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61afe6d-133e-42f4-bc45-5e4835e8e6b9 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:22.710930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4b3a0-77eb-45bd-9a5b-bfb85939f375 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:22.818414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a63f3-fc10-43f6-9105-659074bd419c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:22.820326 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a63f3-fc10-43f6-9105-659074bd419c admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:22.820704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a63f3-fc10-43f6-9105-659074bd419c admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:22.821230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a63f3-fc10-43f6-9105-659074bd419c admin admin] PATH_INFO: `/v3/projects/803aa3005d4b46f9bf0d7366ac575f96/users/edfade728cbd4bc8837740dddcb5e440/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:22.824126 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a63f3-fc10-43f6-9105-659074bd419c admin admin] RBAC: Authorizing `identity:create_grant(project_id=803aa3005d4b46f9bf0d7366ac575f96, user_id=edfade728cbd4bc8837740dddcb5e440, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:22.827717 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a63f3-fc10-43f6-9105-659074bd419c admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:22.833783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61afe6d-133e-42f4-bc45-5e4835e8e6b9 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:22.841427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3367cc-2850-4b1a-a1eb-07daa9e4995c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:22.842387 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3367cc-2850-4b1a-a1eb-07daa9e4995c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:22.843307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3367cc-2850-4b1a-a1eb-07daa9e4995c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:22.843592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3367cc-2850-4b1a-a1eb-07daa9e4995c admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:22.844660 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3367cc-2850-4b1a-a1eb-07daa9e4995c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:22.846940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3367cc-2850-4b1a-a1eb-07daa9e4995c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:22.854335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 806/1635] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:22 2022] PUT /identity/v3/projects/803aa3005d4b46f9bf0d7366ac575f96/users/edfade728cbd4bc8837740dddcb5e440/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:22.864824 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837da10-ce78-4134-b264-468e2845be25 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:22.865120 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837da10-ce78-4134-b264-468e2845be25 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:22.865293 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837da10-ce78-4134-b264-468e2845be25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:22.899687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 830/1636] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:22 2022] POST /identity/v3/users => generated 296 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:32:22.903019 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3367cc-2850-4b1a-a1eb-07daa9e4995c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:22.913818 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3367cc-2850-4b1a-a1eb-07daa9e4995c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:22.915713 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-b837da10-ce78-4134-b264-468e2845be25 None None] MFA Rules not processed for user `edfade728cbd4bc8837740dddcb5e440`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:22.917836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b837da10-ce78-4134-b264-468e2845be25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:22.991486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:23.021142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae82ccb-5aa0-4acb-bc9d-5b4793c8c787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:23.021826 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae82ccb-5aa0-4acb-bc9d-5b4793c8c787 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:23.022030 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae82ccb-5aa0-4acb-bc9d-5b4793c8c787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:23.022305 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae82ccb-5aa0-4acb-bc9d-5b4793c8c787 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:23.022959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae82ccb-5aa0-4acb-bc9d-5b4793c8c787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:23.024606 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae82ccb-5aa0-4acb-bc9d-5b4793c8c787 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:23.051981 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 831/1637] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:32:23.052951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae82ccb-5aa0-4acb-bc9d-5b4793c8c787 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:23.074758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 807/1638] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:22 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:23.088519 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae82ccb-5aa0-4acb-bc9d-5b4793c8c787 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:23.089594 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837da10-ce78-4134-b264-468e2845be25 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:23.105100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f894cfb-0169-486b-97d3-d7c9590d4c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:23.105780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f894cfb-0169-486b-97d3-d7c9590d4c8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:23.106060 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f894cfb-0169-486b-97d3-d7c9590d4c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:23.106335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f894cfb-0169-486b-97d3-d7c9590d4c8c admin admin] PATH_INFO: `/v3/projects/9a29ac3f5f60422abf5f2cc4ab09d151/users/afc0628e8883482aa6ef95cf10ab8b8d/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:23.156035 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f894cfb-0169-486b-97d3-d7c9590d4c8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a29ac3f5f60422abf5f2cc4ab09d151, user_id=afc0628e8883482aa6ef95cf10ab8b8d, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:23.159160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f894cfb-0169-486b-97d3-d7c9590d4c8c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:23.177010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:23.187174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 832/1639] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:23 2022] PUT /identity/v3/projects/9a29ac3f5f60422abf5f2cc4ab09d151/users/afc0628e8883482aa6ef95cf10ab8b8d/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:23.191317 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f894cfb-0169-486b-97d3-d7c9590d4c8c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:23.215165 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837da10-ce78-4134-b264-468e2845be25 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:23.326382 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f894cfb-0169-486b-97d3-d7c9590d4c8c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:23.332988 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033950a6-f6c2-4706-9ca9-39796b47de97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:23.333626 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033950a6-f6c2-4706-9ca9-39796b47de97 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:23.333905 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033950a6-f6c2-4706-9ca9-39796b47de97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:23.334177 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033950a6-f6c2-4706-9ca9-39796b47de97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:23.338376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e0b45-17ff-402b-a61a-f1086c8c2d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:23.339952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e0b45-17ff-402b-a61a-f1086c8c2d06 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:23.340240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e0b45-17ff-402b-a61a-f1086c8c2d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:23.341140 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e0b45-17ff-402b-a61a-f1086c8c2d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:23.341795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e0b45-17ff-402b-a61a-f1086c8c2d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:23.346277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e0b45-17ff-402b-a61a-f1086c8c2d06 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:23.377520 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 833/1640] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:23.378709 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e0b45-17ff-402b-a61a-f1086c8c2d06 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:23.397483 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e0b45-17ff-402b-a61a-f1086c8c2d06 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:23.417857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49091b31-6aa4-4c72-aa7c-ea3bb0f05cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:23.421554 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49091b31-6aa4-4c72-aa7c-ea3bb0f05cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:23.421886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49091b31-6aa4-4c72-aa7c-ea3bb0f05cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:23.422170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49091b31-6aa4-4c72-aa7c-ea3bb0f05cd0 admin admin] PATH_INFO: `/v3/projects/9a29ac3f5f60422abf5f2cc4ab09d151/users/afc0628e8883482aa6ef95cf10ab8b8d/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:23.431964 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49091b31-6aa4-4c72-aa7c-ea3bb0f05cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a29ac3f5f60422abf5f2cc4ab09d151, user_id=afc0628e8883482aa6ef95cf10ab8b8d, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:23.440222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49091b31-6aa4-4c72-aa7c-ea3bb0f05cd0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:23.468506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 834/1641] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:23 2022] PUT /identity/v3/projects/9a29ac3f5f60422abf5f2cc4ab09d151/users/afc0628e8883482aa6ef95cf10ab8b8d/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:23.478760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d1e47-a162-45b1-bf69-dce90f3b401d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:23.479418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d1e47-a162-45b1-bf69-dce90f3b401d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:23.479418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d1e47-a162-45b1-bf69-dce90f3b401d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:23.495760 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033950a6-f6c2-4706-9ca9-39796b47de97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:23.502052 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033950a6-f6c2-4706-9ca9-39796b47de97 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:23.523793 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 808/1642] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:23 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:32:23.542529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-708d1e47-a162-45b1-bf69-dce90f3b401d None None] MFA Rules not processed for user `afc0628e8883482aa6ef95cf10ab8b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:23.543795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-708d1e47-a162-45b1-bf69-dce90f3b401d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:23.656930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f9180-6801-4df4-adfa-1a3e3d11ac00 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:23.657316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f9180-6801-4df4-adfa-1a3e3d11ac00 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:23.657559 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f9180-6801-4df4-adfa-1a3e3d11ac00 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:23.659306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 809/1643] 10.222.0.58 () {54 vars in 962 bytes} [Wed Aug 24 12:32:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:32:23.674864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc721f57-999d-4acc-a56c-e4f6ee38cf39 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:23.675157 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc721f57-999d-4acc-a56c-e4f6ee38cf39 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:23.675496 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc721f57-999d-4acc-a56c-e4f6ee38cf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:23.718416 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 835/1644] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:23.729907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d1e47-a162-45b1-bf69-dce90f3b401d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:23.772598 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-fc721f57-999d-4acc-a56c-e4f6ee38cf39 None None] MFA Rules not processed for user `311dbed3ab0b44a7830a93227ff9ad33`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:23.774271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fc721f57-999d-4acc-a56c-e4f6ee38cf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:23.808212 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:23.891881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d1e47-a162-45b1-bf69-dce90f3b401d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:23.896016 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 810/1645] 10.222.0.58 () {58 vars in 1059 bytes} [Wed Aug 24 12:32:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:23.896993 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc721f57-999d-4acc-a56c-e4f6ee38cf39 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:23.901732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037bf7f1-1a70-4604-ae0a-af2ba9119ed8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:23.902371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bf7f1-1a70-4604-ae0a-af2ba9119ed8 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:23.902653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bf7f1-1a70-4604-ae0a-af2ba9119ed8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:23.902931 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037bf7f1-1a70-4604-ae0a-af2ba9119ed8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:23.930297 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037bf7f1-1a70-4604-ae0a-af2ba9119ed8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:23.931804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037bf7f1-1a70-4604-ae0a-af2ba9119ed8 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:23.945614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:23.958646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 836/1646] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:23.960021 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d161d-d2ff-42be-804b-c71418585a9d None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:23.960315 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d161d-d2ff-42be-804b-c71418585a9d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:23.960725 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d161d-d2ff-42be-804b-c71418585a9d None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:23.961999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 837/1647] 10.222.0.58 () {54 vars in 912 bytes} [Wed Aug 24 12:32:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:32:23.977285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d94173-fc81-4a50-bc37-054fedeb57f7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:23.977578 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d94173-fc81-4a50-bc37-054fedeb57f7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:23.977865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d94173-fc81-4a50-bc37-054fedeb57f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:24.016419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e0d94173-fc81-4a50-bc37-054fedeb57f7 None None] MFA Rules not processed for user `311dbed3ab0b44a7830a93227ff9ad33`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:24.017846 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e0d94173-fc81-4a50-bc37-054fedeb57f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:24.023076 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc721f57-999d-4acc-a56c-e4f6ee38cf39 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:24.028489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180be4d6-d5aa-4042-a7c1-140efe72e3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:24.029196 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180be4d6-d5aa-4042-a7c1-140efe72e3f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:24.029567 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180be4d6-d5aa-4042-a7c1-140efe72e3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:24.030804 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180be4d6-d5aa-4042-a7c1-140efe72e3f6 admin admin] PATH_INFO: `/v3/users/b60bc0c27b494f83a3b342ed407cb6e9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:24.032491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180be4d6-d5aa-4042-a7c1-140efe72e3f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b60bc0c27b494f83a3b342ed407cb6e9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:24.034074 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: WARNING py.warnings [None req-180be4d6-d5aa-4042-a7c1-140efe72e3f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:32:24.034074 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: warnings.warn(msg) Aug 24 12:32:24.034074 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: Aug 24 12:32:24.035951 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180be4d6-d5aa-4042-a7c1-140efe72e3f6 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:24.047540 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 838/1648] 10.222.0.58 () {58 vars in 1009 bytes} [Wed Aug 24 12:32:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:24.048762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d94173-fc81-4a50-bc37-054fedeb57f7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:24.068648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d94173-fc81-4a50-bc37-054fedeb57f7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:24.082276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075276b5-90cf-44c4-aebd-5c0a3830a744 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:24.082613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075276b5-90cf-44c4-aebd-5c0a3830a744 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:24.082897 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075276b5-90cf-44c4-aebd-5c0a3830a744 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:24.083172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075276b5-90cf-44c4-aebd-5c0a3830a744 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:24.159292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-180be4d6-d5aa-4042-a7c1-140efe72e3f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b60bc0c27b494f83a3b342ed407cb6e9'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:24.160073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-180be4d6-d5aa-4042-a7c1-140efe72e3f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b60bc0c27b494f83a3b342ed407cb6e9'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:24.161031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-180be4d6-d5aa-4042-a7c1-140efe72e3f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b60bc0c27b494f83a3b342ed407cb6e9'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:24.182311 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-180be4d6-d5aa-4042-a7c1-140efe72e3f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b60bc0c27b494f83a3b342ed407cb6e9'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:24.205004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-180be4d6-d5aa-4042-a7c1-140efe72e3f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b60bc0c27b494f83a3b342ed407cb6e9'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:24.234684 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075276b5-90cf-44c4-aebd-5c0a3830a744 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:24.238183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075276b5-90cf-44c4-aebd-5c0a3830a744 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:24.268345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 839/1649] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:24.271009 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075276b5-90cf-44c4-aebd-5c0a3830a744 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:24.280314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 811/1650] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:23 2022] DELETE /identity/v3/users/b60bc0c27b494f83a3b342ed407cb6e9 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:24.283308 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109d2d5-ee81-41e8-b404-f1e0e95298ae None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:24.283556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109d2d5-ee81-41e8-b404-f1e0e95298ae None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:24.283859 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109d2d5-ee81-41e8-b404-f1e0e95298ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:24.295331 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075276b5-90cf-44c4-aebd-5c0a3830a744 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:24.329503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-c109d2d5-ee81-41e8-b404-f1e0e95298ae None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:24.333400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c109d2d5-ee81-41e8-b404-f1e0e95298ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:24.441636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592fdd9-9ae2-4e17-964b-7572f91ae499 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:24.443747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592fdd9-9ae2-4e17-964b-7572f91ae499 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:24.446183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592fdd9-9ae2-4e17-964b-7572f91ae499 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:24.446472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592fdd9-9ae2-4e17-964b-7572f91ae499 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:24.448963 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1592fdd9-9ae2-4e17-964b-7572f91ae499 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:24.513781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 812/1651] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:24.545141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109d2d5-ee81-41e8-b404-f1e0e95298ae None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:24.547729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c109d2d5-ee81-41e8-b404-f1e0e95298ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:24.645731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592fdd9-9ae2-4e17-964b-7572f91ae499 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:24.648184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592fdd9-9ae2-4e17-964b-7572f91ae499 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:24.683388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 840/1652] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:24.685188 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109d2d5-ee81-41e8-b404-f1e0e95298ae None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:24.704769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b3cb9-46ad-4c73-a709-2d599e763201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:24.705721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b3cb9-46ad-4c73-a709-2d599e763201 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:24.706007 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b3cb9-46ad-4c73-a709-2d599e763201 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:24.707780 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b3cb9-46ad-4c73-a709-2d599e763201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:24.731250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:24.738644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b3cb9-46ad-4c73-a709-2d599e763201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:24.743884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477b3cb9-46ad-4c73-a709-2d599e763201 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:24.779770 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 813/1653] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:26.357913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592fdd9-9ae2-4e17-964b-7572f91ae499 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:26.357913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1592fdd9-9ae2-4e17-964b-7572f91ae499 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:26.366451 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:26.392189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592fdd9-9ae2-4e17-964b-7572f91ae499 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:26.409281 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1014830-3e12-4f4b-b228-a49edde7add2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:26.409941 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1014830-3e12-4f4b-b228-a49edde7add2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:26.410218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1014830-3e12-4f4b-b228-a49edde7add2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:26.410488 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1014830-3e12-4f4b-b228-a49edde7add2 admin admin] PATH_INFO: `/v3/users/8c4727a0ef41431992676f231464f041` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:26.411672 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1014830-3e12-4f4b-b228-a49edde7add2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c4727a0ef41431992676f231464f041)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:26.414831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1014830-3e12-4f4b-b228-a49edde7add2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:26.430482 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b3cb9-46ad-4c73-a709-2d599e763201 service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:26.432237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-477b3cb9-46ad-4c73-a709-2d599e763201 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:26.462832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477b3cb9-46ad-4c73-a709-2d599e763201 service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:26.476805 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c31ee-1073-4c80-9461-49f51bd71e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:26.477762 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c31ee-1073-4c80-9461-49f51bd71e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:26.478226 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c31ee-1073-4c80-9461-49f51bd71e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:26.478652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c31ee-1073-4c80-9461-49f51bd71e6f admin admin] PATH_INFO: `/v3/projects/3a1d2aaa6b3a44a9a9e7daabd3c7a028` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:26.487972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155c31ee-1073-4c80-9461-49f51bd71e6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a1d2aaa6b3a44a9a9e7daabd3c7a028)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:26.490314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155c31ee-1073-4c80-9461-49f51bd71e6f admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:26.570740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e1014830-3e12-4f4b-b228-a49edde7add2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c4727a0ef41431992676f231464f041'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:26.572482 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e1014830-3e12-4f4b-b228-a49edde7add2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c4727a0ef41431992676f231464f041'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:26.582132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e1014830-3e12-4f4b-b228-a49edde7add2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c4727a0ef41431992676f231464f041'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:26.585598 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-155c31ee-1073-4c80-9461-49f51bd71e6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a1d2aaa6b3a44a9a9e7daabd3c7a028', 'request_id': 'req-155c31ee-1073-4c80-9461-49f51bd71e6f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:26.632387 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-155c31ee-1073-4c80-9461-49f51bd71e6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a1d2aaa6b3a44a9a9e7daabd3c7a028', 'request_id': 'req-155c31ee-1073-4c80-9461-49f51bd71e6f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:26.633756 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e1014830-3e12-4f4b-b228-a49edde7add2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c4727a0ef41431992676f231464f041'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:26.634030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-155c31ee-1073-4c80-9461-49f51bd71e6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a1d2aaa6b3a44a9a9e7daabd3c7a028', 'request_id': 'req-155c31ee-1073-4c80-9461-49f51bd71e6f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:26.641561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e1014830-3e12-4f4b-b228-a49edde7add2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c4727a0ef41431992676f231464f041'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:26.650548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 841/1654] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:26 2022] DELETE /identity/v3/users/8c4727a0ef41431992676f231464f041 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:26.655537 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cfee5-c3fb-45da-a841-9d672a36f84c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:26.656149 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cfee5-c3fb-45da-a841-9d672a36f84c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:26.656610 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52cfee5-c3fb-45da-a841-9d672a36f84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:26.689114 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-155c31ee-1073-4c80-9461-49f51bd71e6f admin admin] The token cache is being invalidate because project 3a1d2aaa6b3a44a9a9e7daabd3c7a028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:26.689418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-155c31ee-1073-4c80-9461-49f51bd71e6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:26.690154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-155c31ee-1073-4c80-9461-49f51bd71e6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:26.695333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 814/1655] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:26 2022] DELETE /identity/v3/projects/3a1d2aaa6b3a44a9a9e7daabd3c7a028 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:26.696415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c31ee-1073-4c80-9461-49f51bd71e6f admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:26.697848 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-155c31ee-1073-4c80-9461-49f51bd71e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:26.726658 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f52cfee5-c3fb-45da-a841-9d672a36f84c None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:26.730180 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f52cfee5-c3fb-45da-a841-9d672a36f84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:26.889631 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c31ee-1073-4c80-9461-49f51bd71e6f admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:26.955052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 842/1656] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:27.022939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465ef13-debc-4174-96ba-fb1f5b523495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:27.023602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465ef13-debc-4174-96ba-fb1f5b523495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:27.023922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465ef13-debc-4174-96ba-fb1f5b523495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:27.024201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8465ef13-debc-4174-96ba-fb1f5b523495 admin admin] PATH_INFO: `/v3/users/edfade728cbd4bc8837740dddcb5e440` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:27.025805 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465ef13-debc-4174-96ba-fb1f5b523495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edfade728cbd4bc8837740dddcb5e440)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:27.028140 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8465ef13-debc-4174-96ba-fb1f5b523495 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:27.075871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:27.123227 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-8465ef13-debc-4174-96ba-fb1f5b523495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edfade728cbd4bc8837740dddcb5e440'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:27.124535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-8465ef13-debc-4174-96ba-fb1f5b523495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edfade728cbd4bc8837740dddcb5e440'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:27.127835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-8465ef13-debc-4174-96ba-fb1f5b523495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edfade728cbd4bc8837740dddcb5e440'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:27.152009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-8465ef13-debc-4174-96ba-fb1f5b523495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edfade728cbd4bc8837740dddcb5e440'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:27.157389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cfee5-c3fb-45da-a841-9d672a36f84c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:27.159068 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f52cfee5-c3fb-45da-a841-9d672a36f84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:27.173498 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-8465ef13-debc-4174-96ba-fb1f5b523495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edfade728cbd4bc8837740dddcb5e440'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:27.210362 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 815/1657] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:26 2022] DELETE /identity/v3/users/edfade728cbd4bc8837740dddcb5e440 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:27.339985 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52cfee5-c3fb-45da-a841-9d672a36f84c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:27.397884 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:27.447314 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d693e-2ef1-4bef-aa62-84ff0d0e964c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:27.447977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d693e-2ef1-4bef-aa62-84ff0d0e964c service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:27.448440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d693e-2ef1-4bef-aa62-84ff0d0e964c service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:27.448868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d693e-2ef1-4bef-aa62-84ff0d0e964c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:27.453238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cc1d693e-2ef1-4bef-aa62-84ff0d0e964c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:27.612010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d693e-2ef1-4bef-aa62-84ff0d0e964c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:27.614693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1d693e-2ef1-4bef-aa62-84ff0d0e964c service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:27.630140 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 843/1658] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:28.959667 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465ef13-debc-4174-96ba-fb1f5b523495 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:28.961155 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-8465ef13-debc-4174-96ba-fb1f5b523495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:28.997579 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8465ef13-debc-4174-96ba-fb1f5b523495 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:29.009063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19328be-1800-444d-a45f-26b6306a7561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:29.016959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19328be-1800-444d-a45f-26b6306a7561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:29.017512 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19328be-1800-444d-a45f-26b6306a7561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:29.017800 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19328be-1800-444d-a45f-26b6306a7561 admin admin] PATH_INFO: `/v3/users/9bf0e4a8956744a2aff4a3c2dea24b26` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:29.019709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19328be-1800-444d-a45f-26b6306a7561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf0e4a8956744a2aff4a3c2dea24b26)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:29.021863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19328be-1800-444d-a45f-26b6306a7561 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:29.163198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c19328be-1800-444d-a45f-26b6306a7561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf0e4a8956744a2aff4a3c2dea24b26'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:29.164473 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c19328be-1800-444d-a45f-26b6306a7561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf0e4a8956744a2aff4a3c2dea24b26'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:29.164633 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c19328be-1800-444d-a45f-26b6306a7561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf0e4a8956744a2aff4a3c2dea24b26'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:29.188895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c19328be-1800-444d-a45f-26b6306a7561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf0e4a8956744a2aff4a3c2dea24b26'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:29.237602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c19328be-1800-444d-a45f-26b6306a7561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf0e4a8956744a2aff4a3c2dea24b26'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:29.302502 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 816/1659] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:28 2022] DELETE /identity/v3/users/9bf0e4a8956744a2aff4a3c2dea24b26 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:29.306085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5821a-0b2b-402c-88fa-83e07a417a80 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:29.306624 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5821a-0b2b-402c-88fa-83e07a417a80 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:29.306624 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f5821a-0b2b-402c-88fa-83e07a417a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:29.328899 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-65f5821a-0b2b-402c-88fa-83e07a417a80 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:29.330133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-65f5821a-0b2b-402c-88fa-83e07a417a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:29.449373 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 844/1660] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:29.506470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19328be-1800-444d-a45f-26b6306a7561 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:29.508449 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c19328be-1800-444d-a45f-26b6306a7561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:29.645069 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19328be-1800-444d-a45f-26b6306a7561 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:29.651898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1fef9-e2cf-466a-a36e-7e3b2d22e38c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:29.652664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1fef9-e2cf-466a-a36e-7e3b2d22e38c service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:29.653109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1fef9-e2cf-466a-a36e-7e3b2d22e38c service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:29.653486 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1fef9-e2cf-466a-a36e-7e3b2d22e38c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:29.666451 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1fef9-e2cf-466a-a36e-7e3b2d22e38c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:29.668360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1fef9-e2cf-466a-a36e-7e3b2d22e38c service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:29.689533 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 817/1661] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:29.862839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5821a-0b2b-402c-88fa-83e07a417a80 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:29.864387 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-65f5821a-0b2b-402c-88fa-83e07a417a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:29.886233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f5821a-0b2b-402c-88fa-83e07a417a80 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:29.897425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc9d21-0775-4ce3-96d0-3ece024d7d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:29.898858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc9d21-0775-4ce3-96d0-3ece024d7d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:29.899176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc9d21-0775-4ce3-96d0-3ece024d7d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:29.899414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc9d21-0775-4ce3-96d0-3ece024d7d73 admin admin] PATH_INFO: `/v3/projects/803aa3005d4b46f9bf0d7366ac575f96` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:29.903780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc9d21-0775-4ce3-96d0-3ece024d7d73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=803aa3005d4b46f9bf0d7366ac575f96)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:29.906413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: WARNING py.warnings [None req-eefc9d21-0775-4ce3-96d0-3ece024d7d73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 12:32:29.906413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: warnings.warn(msg) Aug 24 12:32:29.906413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: Aug 24 12:32:29.907502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc9d21-0775-4ce3-96d0-3ece024d7d73 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:30.029079 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-eefc9d21-0775-4ce3-96d0-3ece024d7d73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '803aa3005d4b46f9bf0d7366ac575f96', 'request_id': 'req-eefc9d21-0775-4ce3-96d0-3ece024d7d73'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:30.045277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-eefc9d21-0775-4ce3-96d0-3ece024d7d73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '803aa3005d4b46f9bf0d7366ac575f96', 'request_id': 'req-eefc9d21-0775-4ce3-96d0-3ece024d7d73'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:30.062631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-eefc9d21-0775-4ce3-96d0-3ece024d7d73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '803aa3005d4b46f9bf0d7366ac575f96', 'request_id': 'req-eefc9d21-0775-4ce3-96d0-3ece024d7d73'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:30.107521 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-eefc9d21-0775-4ce3-96d0-3ece024d7d73 admin admin] The token cache is being invalidate because project 803aa3005d4b46f9bf0d7366ac575f96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:30.107987 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-eefc9d21-0775-4ce3-96d0-3ece024d7d73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:30.109297 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-eefc9d21-0775-4ce3-96d0-3ece024d7d73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:30.114624 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 845/1662] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:29 2022] DELETE /identity/v3/projects/803aa3005d4b46f9bf0d7366ac575f96 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:30.116397 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa9581-ef65-411b-b882-377f48e52941 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.116927 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa9581-ef65-411b-b882-377f48e52941 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.117712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa9581-ef65-411b-b882-377f48e52941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.170764 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-47aa9581-ef65-411b-b882-377f48e52941 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:30.172380 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-47aa9581-ef65-411b-b882-377f48e52941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:30.238004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:30.341660 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 818/1663] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:30.352439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75a085-5d18-44b7-93f9-246586198f1a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.353650 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75a085-5d18-44b7-93f9-246586198f1a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.354122 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75a085-5d18-44b7-93f9-246586198f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.369594 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:30.395102 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ab75a085-5d18-44b7-93f9-246586198f1a None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:30.396981 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ab75a085-5d18-44b7-93f9-246586198f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:30.417516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa9581-ef65-411b-b882-377f48e52941 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:30.422292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 846/1664] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:30.432232 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa9581-ef65-411b-b882-377f48e52941 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:30.447772 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd304d-730d-406f-a8c2-43506eaf3d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:30.448730 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd304d-730d-406f-a8c2-43506eaf3d95 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.449244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd304d-730d-406f-a8c2-43506eaf3d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.449649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd304d-730d-406f-a8c2-43506eaf3d95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.450756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd304d-730d-406f-a8c2-43506eaf3d95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:30.453309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd304d-730d-406f-a8c2-43506eaf3d95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:30.457972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd304d-730d-406f-a8c2-43506eaf3d95 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:30.497391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 819/1665] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:32:30.498261 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75a085-5d18-44b7-93f9-246586198f1a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:30.515650 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75a085-5d18-44b7-93f9-246586198f1a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:30.522261 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d08ea-de0d-498f-9222-3b2b8c9549af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:30.525495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d08ea-de0d-498f-9222-3b2b8c9549af admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.525799 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d08ea-de0d-498f-9222-3b2b8c9549af admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.526088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d08ea-de0d-498f-9222-3b2b8c9549af admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.538181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d08ea-de0d-498f-9222-3b2b8c9549af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:30.540712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d08ea-de0d-498f-9222-3b2b8c9549af admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:30.622959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 847/1666] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:30 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:30.623164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd304d-730d-406f-a8c2-43506eaf3d95 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:30.637963 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd304d-730d-406f-a8c2-43506eaf3d95 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:30.771878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796ea1d-b2b9-4c8b-847d-0983c6871fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:30.773712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796ea1d-b2b9-4c8b-847d-0983c6871fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.777502 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796ea1d-b2b9-4c8b-847d-0983c6871fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.778450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796ea1d-b2b9-4c8b-847d-0983c6871fc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.779445 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796ea1d-b2b9-4c8b-847d-0983c6871fc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:30.784989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796ea1d-b2b9-4c8b-847d-0983c6871fc4 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:30.837632 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d08ea-de0d-498f-9222-3b2b8c9549af admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:30.842306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 820/1667] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:30 2022] POST /identity/v3/users => generated 287 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:30.854990 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d08ea-de0d-498f-9222-3b2b8c9549af admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:30.867188 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e8543-e9e1-4e91-877b-e586837f61d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:30.867941 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e8543-e9e1-4e91-877b-e586837f61d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.868308 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e8543-e9e1-4e91-877b-e586837f61d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.869126 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e8543-e9e1-4e91-877b-e586837f61d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.870753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e8543-e9e1-4e91-877b-e586837f61d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:30.875921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e8543-e9e1-4e91-877b-e586837f61d6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:30.905078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796ea1d-b2b9-4c8b-847d-0983c6871fc4 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:30.909278 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 848/1668] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:30.931449 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796ea1d-b2b9-4c8b-847d-0983c6871fc4 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:30.933691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e8543-e9e1-4e91-877b-e586837f61d6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:30.938593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-274e8543-e9e1-4e91-877b-e586837f61d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:30.955244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f1f54-6eaa-4757-b255-2d4bbe87ea4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:30.964284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f1f54-6eaa-4757-b255-2d4bbe87ea4a admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.964571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f1f54-6eaa-4757-b255-2d4bbe87ea4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.964848 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f1f54-6eaa-4757-b255-2d4bbe87ea4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.965571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f1f54-6eaa-4757-b255-2d4bbe87ea4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:30.967148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14f1f54-6eaa-4757-b255-2d4bbe87ea4a admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:30.977685 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e8543-e9e1-4e91-877b-e586837f61d6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:30.986500 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da296d7e-0277-4412-b7e2-80b590e1d219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:30.987813 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da296d7e-0277-4412-b7e2-80b590e1d219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:30.988102 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da296d7e-0277-4412-b7e2-80b590e1d219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:30.988371 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 821/1669] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:30.988482 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da296d7e-0277-4412-b7e2-80b590e1d219 admin admin] PATH_INFO: `/v3/projects/7b5a8391b9c34b80b3cd6f6408c0e13b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:30.993194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f1f54-6eaa-4757-b255-2d4bbe87ea4a admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:30.996799 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da296d7e-0277-4412-b7e2-80b590e1d219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b5a8391b9c34b80b3cd6f6408c0e13b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:30.999907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da296d7e-0277-4412-b7e2-80b590e1d219 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:31.011756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f1f54-6eaa-4757-b255-2d4bbe87ea4a admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:31.022072 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2dd700-8e0d-4d11-a977-508cc9157c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:31.022861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dd700-8e0d-4d11-a977-508cc9157c68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:31.023142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dd700-8e0d-4d11-a977-508cc9157c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:31.023416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2dd700-8e0d-4d11-a977-508cc9157c68 admin admin] PATH_INFO: `/v3/projects/5b91c5b04b42452789555241d7ae1faf/users/4ed8274584ca409bafc99b3e6e6a1ec1/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:31.056510 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2dd700-8e0d-4d11-a977-508cc9157c68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b91c5b04b42452789555241d7ae1faf, user_id=4ed8274584ca409bafc99b3e6e6a1ec1, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:31.059196 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2dd700-8e0d-4d11-a977-508cc9157c68 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:31.093293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da296d7e-0277-4412-b7e2-80b590e1d219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b5a8391b9c34b80b3cd6f6408c0e13b', 'request_id': 'req-da296d7e-0277-4412-b7e2-80b590e1d219'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:31.104166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da296d7e-0277-4412-b7e2-80b590e1d219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b5a8391b9c34b80b3cd6f6408c0e13b', 'request_id': 'req-da296d7e-0277-4412-b7e2-80b590e1d219'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:31.109764 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 822/1670] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:30 2022] PUT /identity/v3/projects/5b91c5b04b42452789555241d7ae1faf/users/4ed8274584ca409bafc99b3e6e6a1ec1/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:31.122374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7faa6-c637-46a7-b127-95fbe93fe942 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:31.122950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7faa6-c637-46a7-b127-95fbe93fe942 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:31.122950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7faa6-c637-46a7-b127-95fbe93fe942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:31.136570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da296d7e-0277-4412-b7e2-80b590e1d219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b5a8391b9c34b80b3cd6f6408c0e13b', 'request_id': 'req-da296d7e-0277-4412-b7e2-80b590e1d219'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:31.167719 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da296d7e-0277-4412-b7e2-80b590e1d219 admin admin] The token cache is being invalidate because project 7b5a8391b9c34b80b3cd6f6408c0e13b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:31.168029 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da296d7e-0277-4412-b7e2-80b590e1d219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:31.171125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da296d7e-0277-4412-b7e2-80b590e1d219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:31.180102 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 849/1671] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:30 2022] DELETE /identity/v3/projects/7b5a8391b9c34b80b3cd6f6408c0e13b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:31.180837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73d562-7f07-4862-941b-79634005b712 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:31.181180 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73d562-7f07-4862-941b-79634005b712 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:31.181507 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73d562-7f07-4862-941b-79634005b712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:31.198056 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-b3b7faa6-c637-46a7-b127-95fbe93fe942 None None] MFA Rules not processed for user `4ed8274584ca409bafc99b3e6e6a1ec1`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:31.199304 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b3b7faa6-c637-46a7-b127-95fbe93fe942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:31.223696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4a73d562-7f07-4862-941b-79634005b712 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:31.226735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4a73d562-7f07-4862-941b-79634005b712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:31.279228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:31.325860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 823/1672] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:31.326629 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7faa6-c637-46a7-b127-95fbe93fe942 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:31.330958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b3b7faa6-c637-46a7-b127-95fbe93fe942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:31.416715 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 850/1673] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:31.417232 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992e07a-7eac-4ad2-825a-eccdf5135e10 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:31.417232 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992e07a-7eac-4ad2-825a-eccdf5135e10 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:31.417907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e992e07a-7eac-4ad2-825a-eccdf5135e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:31.490539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e992e07a-7eac-4ad2-825a-eccdf5135e10 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:31.491821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e992e07a-7eac-4ad2-825a-eccdf5135e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:31.536424 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 851/1674] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:31.537268 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992e07a-7eac-4ad2-825a-eccdf5135e10 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:31.565935 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7faa6-c637-46a7-b127-95fbe93fe942 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:31.582300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e992e07a-7eac-4ad2-825a-eccdf5135e10 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:31.597538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25589f5-9114-4874-a845-c3c4082366ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:31.598190 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25589f5-9114-4874-a845-c3c4082366ea admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:31.598474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25589f5-9114-4874-a845-c3c4082366ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:31.598747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25589f5-9114-4874-a845-c3c4082366ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:31.602708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25589f5-9114-4874-a845-c3c4082366ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:31.609166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bd671-b776-483c-9f82-22d2c2fff817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:31.610319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bd671-b776-483c-9f82-22d2c2fff817 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:31.610659 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bd671-b776-483c-9f82-22d2c2fff817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:31.610857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bd671-b776-483c-9f82-22d2c2fff817 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:31.613360 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9bd671-b776-483c-9f82-22d2c2fff817 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:31.613360 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9bd671-b776-483c-9f82-22d2c2fff817 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:31.615587 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25589f5-9114-4874-a845-c3c4082366ea admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:31.626506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9bd671-b776-483c-9f82-22d2c2fff817 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:31.646426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 852/1675] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:31.651073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bd671-b776-483c-9f82-22d2c2fff817 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:31.666029 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bd671-b776-483c-9f82-22d2c2fff817 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:31.676163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb321c8e-f3ae-4fad-977f-6e7e2e29a004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:31.677056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb321c8e-f3ae-4fad-977f-6e7e2e29a004 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:31.677704 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb321c8e-f3ae-4fad-977f-6e7e2e29a004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:31.678252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb321c8e-f3ae-4fad-977f-6e7e2e29a004 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:31.679101 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb321c8e-f3ae-4fad-977f-6e7e2e29a004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:31.682490 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb321c8e-f3ae-4fad-977f-6e7e2e29a004 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:31.726789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 824/1676] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:31 2022] POST /identity/v3/projects => generated 373 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:32:31.727685 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25589f5-9114-4874-a845-c3c4082366ea admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:31.772955 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 853/1677] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:31 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:31.781906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb321c8e-f3ae-4fad-977f-6e7e2e29a004 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:31.814653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:31.956666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25589f5-9114-4874-a845-c3c4082366ea admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:31.993710 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb321c8e-f3ae-4fad-977f-6e7e2e29a004 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:32.003771 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f5935-9215-4324-99ad-cb93702e335b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:32.004403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f5935-9215-4324-99ad-cb93702e335b admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:32.004677 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f5935-9215-4324-99ad-cb93702e335b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:32.004946 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f5935-9215-4324-99ad-cb93702e335b admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:32.008334 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f5935-9215-4324-99ad-cb93702e335b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:32.013001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f5935-9215-4324-99ad-cb93702e335b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:32.077843 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 854/1678] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:31 2022] POST /identity/v3/users => generated 286 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:32.081710 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f5935-9215-4324-99ad-cb93702e335b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:32.101752 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f5935-9215-4324-99ad-cb93702e335b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:32.106576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712b10f-f890-4010-896e-967d10e91411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:32.107582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712b10f-f890-4010-896e-967d10e91411 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:32.107928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712b10f-f890-4010-896e-967d10e91411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:32.108822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712b10f-f890-4010-896e-967d10e91411 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:32.109416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712b10f-f890-4010-896e-967d10e91411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:32.112477 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712b10f-f890-4010-896e-967d10e91411 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:32.115875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4a8d9-ba65-4b1d-b921-b8dd01836c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:32.116504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4a8d9-ba65-4b1d-b921-b8dd01836c9c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:32.116782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4a8d9-ba65-4b1d-b921-b8dd01836c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:32.117052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4a8d9-ba65-4b1d-b921-b8dd01836c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:32.117718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed4a8d9-ba65-4b1d-b921-b8dd01836c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:32.121580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed4a8d9-ba65-4b1d-b921-b8dd01836c9c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:32.148674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 855/1679] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:32.151165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4a8d9-ba65-4b1d-b921-b8dd01836c9c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:32.172436 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 825/1680] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:31 2022] POST /identity/v3/users => generated 293 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:32.177127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712b10f-f890-4010-896e-967d10e91411 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:32.178403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4a8d9-ba65-4b1d-b921-b8dd01836c9c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:32.189269 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f9dfc-8bdd-4b71-b356-e6008d82f754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:32.192350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f9dfc-8bdd-4b71-b356-e6008d82f754 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:32.192638 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f9dfc-8bdd-4b71-b356-e6008d82f754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:32.192913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f9dfc-8bdd-4b71-b356-e6008d82f754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:32.193580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f9dfc-8bdd-4b71-b356-e6008d82f754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:32.198047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712b10f-f890-4010-896e-967d10e91411 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:32.198807 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f9dfc-8bdd-4b71-b356-e6008d82f754 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:32.206965 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9a470-4c00-4e60-b59c-c324da6d7ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:32.207592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9a470-4c00-4e60-b59c-c324da6d7ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:32.213524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9a470-4c00-4e60-b59c-c324da6d7ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:32.213867 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f9a470-4c00-4e60-b59c-c324da6d7ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:32.214618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9a470-4c00-4e60-b59c-c324da6d7ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:32.217198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f9a470-4c00-4e60-b59c-c324da6d7ac5 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:32.229859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 856/1681] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:32.232714 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f9dfc-8bdd-4b71-b356-e6008d82f754 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:32.243309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 826/1682] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:32.244208 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9a470-4c00-4e60-b59c-c324da6d7ac5 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:32.253036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:32.259001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f9dfc-8bdd-4b71-b356-e6008d82f754 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:32.268795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbb7ae-955b-4ea1-a937-e33de62ec22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:32.269695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbb7ae-955b-4ea1-a937-e33de62ec22c admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:32.270112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f9a470-4c00-4e60-b59c-c324da6d7ac5 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:32.272494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbb7ae-955b-4ea1-a937-e33de62ec22c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:32.272786 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbb7ae-955b-4ea1-a937-e33de62ec22c admin admin] PATH_INFO: `/v3/projects/3d86d0b1ee1c4bea84c6511ef8c57d33/users/840b4225d8304fd1919cdeafb15cf901/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:32.283023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f1b81-6b61-4d11-bb90-c90c103ab663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:32.284074 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f1b81-6b61-4d11-bb90-c90c103ab663 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:32.284366 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f1b81-6b61-4d11-bb90-c90c103ab663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:32.284643 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f1b81-6b61-4d11-bb90-c90c103ab663 admin admin] PATH_INFO: `/v3/projects/c41697d4a34f4c7fb6e8bbc4b4cf5933/users/f2456a6677fa4408a91fd4c3a2488ebf/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:32.311496 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbb7ae-955b-4ea1-a937-e33de62ec22c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d86d0b1ee1c4bea84c6511ef8c57d33, user_id=840b4225d8304fd1919cdeafb15cf901, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:32.315132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbb7ae-955b-4ea1-a937-e33de62ec22c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:32.322690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f1b81-6b61-4d11-bb90-c90c103ab663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c41697d4a34f4c7fb6e8bbc4b4cf5933, user_id=f2456a6677fa4408a91fd4c3a2488ebf, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:32.326513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033f1b81-6b61-4d11-bb90-c90c103ab663 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:32.364040 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 827/1683] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:32 2022] PUT /identity/v3/projects/c41697d4a34f4c7fb6e8bbc4b4cf5933/users/f2456a6677fa4408a91fd4c3a2488ebf/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:32.367521 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 857/1684] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:32 2022] PUT /identity/v3/projects/3d86d0b1ee1c4bea84c6511ef8c57d33/users/840b4225d8304fd1919cdeafb15cf901/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:32.369341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f1b81-6b61-4d11-bb90-c90c103ab663 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:32.374853 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fce00-0340-47a3-9bb4-ec3b08301875 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:32.375135 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fce00-0340-47a3-9bb4-ec3b08301875 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:32.375408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fce00-0340-47a3-9bb4-ec3b08301875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:32.465430 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-569fce00-0340-47a3-9bb4-ec3b08301875 None None] MFA Rules not processed for user `840b4225d8304fd1919cdeafb15cf901`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:32.466677 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-569fce00-0340-47a3-9bb4-ec3b08301875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:32.536237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f1b81-6b61-4d11-bb90-c90c103ab663 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:32.541879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62482e61-fc4e-4db0-8273-c3cd4cdedb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:32.542538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62482e61-fc4e-4db0-8273-c3cd4cdedb3e admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:32.542986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62482e61-fc4e-4db0-8273-c3cd4cdedb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:32.543261 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62482e61-fc4e-4db0-8273-c3cd4cdedb3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:32.543964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62482e61-fc4e-4db0-8273-c3cd4cdedb3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:32.547016 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62482e61-fc4e-4db0-8273-c3cd4cdedb3e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:32.575504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 828/1685] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:32 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:32.581656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62482e61-fc4e-4db0-8273-c3cd4cdedb3e admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:32.605720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62482e61-fc4e-4db0-8273-c3cd4cdedb3e admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:32.631208 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6300d-b693-46c6-828f-ee6513511ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:32.632341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6300d-b693-46c6-828f-ee6513511ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:32.632865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6300d-b693-46c6-828f-ee6513511ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:32.633316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6300d-b693-46c6-828f-ee6513511ed2 admin admin] PATH_INFO: `/v3/projects/c41697d4a34f4c7fb6e8bbc4b4cf5933/users/f2456a6677fa4408a91fd4c3a2488ebf/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:32.636359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6300d-b693-46c6-828f-ee6513511ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c41697d4a34f4c7fb6e8bbc4b4cf5933, user_id=f2456a6677fa4408a91fd4c3a2488ebf, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:32.646293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 858/1686] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:32.649890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6300d-b693-46c6-828f-ee6513511ed2 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:32.653339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fce00-0340-47a3-9bb4-ec3b08301875 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:32.665625 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 829/1687] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:32 2022] PUT /identity/v3/projects/c41697d4a34f4c7fb6e8bbc4b4cf5933/users/f2456a6677fa4408a91fd4c3a2488ebf/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:32.677978 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b990f-0022-41ba-8370-9a190a55bfc2 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:32.680467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b990f-0022-41ba-8370-9a190a55bfc2 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:32.681093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b990f-0022-41ba-8370-9a190a55bfc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:32.688779 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fce00-0340-47a3-9bb4-ec3b08301875 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:32.756538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-1b0b990f-0022-41ba-8370-9a190a55bfc2 None None] MFA Rules not processed for user `f2456a6677fa4408a91fd4c3a2488ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:32.760506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1b0b990f-0022-41ba-8370-9a190a55bfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:32.805491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:32.856077 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e257f88-4c69-4f45-baf0-e7434271ba29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:32.856713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e257f88-4c69-4f45-baf0-e7434271ba29 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:32.857149 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e257f88-4c69-4f45-baf0-e7434271ba29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:32.857436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e257f88-4c69-4f45-baf0-e7434271ba29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:32.861337 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e257f88-4c69-4f45-baf0-e7434271ba29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:32.863376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e257f88-4c69-4f45-baf0-e7434271ba29 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:32.927758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 859/1688] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:32 2022] POST /identity/v3/projects => generated 371 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:32.932171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e257f88-4c69-4f45-baf0-e7434271ba29 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:32.965979 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 830/1689] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:32.984954 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b990f-0022-41ba-8370-9a190a55bfc2 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:32.987135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1b0b990f-0022-41ba-8370-9a190a55bfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:33.116861 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e257f88-4c69-4f45-baf0-e7434271ba29 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.118632 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:33.129409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b990f-0022-41ba-8370-9a190a55bfc2 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.134307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a36ed-d5f9-41c8-b35c-c3fcab29e6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.134952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a36ed-d5f9-41c8-b35c-c3fcab29e6df admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.135289 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a36ed-d5f9-41c8-b35c-c3fcab29e6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.135502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a36ed-d5f9-41c8-b35c-c3fcab29e6df admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.137438 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a36ed-d5f9-41c8-b35c-c3fcab29e6df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.144301 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a36ed-d5f9-41c8-b35c-c3fcab29e6df admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.192642 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 860/1690] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:32 2022] POST /identity/v3/users => generated 292 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:33.196316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0459cd1b-4247-4de7-975b-652a23824bc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.196954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a36ed-d5f9-41c8-b35c-c3fcab29e6df admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.197107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459cd1b-4247-4de7-975b-652a23824bc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.197301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459cd1b-4247-4de7-975b-652a23824bc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.197685 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0459cd1b-4247-4de7-975b-652a23824bc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.225173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a36ed-d5f9-41c8-b35c-c3fcab29e6df admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.236226 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d67345-4203-4e6d-992c-2ef8a5005ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.236958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d67345-4203-4e6d-992c-2ef8a5005ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.237250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d67345-4203-4e6d-992c-2ef8a5005ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.237530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d67345-4203-4e6d-992c-2ef8a5005ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.238876 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d67345-4203-4e6d-992c-2ef8a5005ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.241040 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d67345-4203-4e6d-992c-2ef8a5005ef0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.271833 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 861/1691] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:33.273740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d67345-4203-4e6d-992c-2ef8a5005ef0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.291198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0459cd1b-4247-4de7-975b-652a23824bc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.293107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0459cd1b-4247-4de7-975b-652a23824bc2 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.297749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d67345-4203-4e6d-992c-2ef8a5005ef0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.304919 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7fd3ac-b186-46b0-9214-b282e80e754b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.305779 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7fd3ac-b186-46b0-9214-b282e80e754b admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.306130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7fd3ac-b186-46b0-9214-b282e80e754b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.306529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7fd3ac-b186-46b0-9214-b282e80e754b admin admin] PATH_INFO: `/v3/projects/82ce5689ca214792a7ab40f4905c47f6/users/b29cf33afd1443b68b49d5b5f4044c69/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.315206 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 831/1692] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:32:33.352930 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7fd3ac-b186-46b0-9214-b282e80e754b admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ce5689ca214792a7ab40f4905c47f6, user_id=b29cf33afd1443b68b49d5b5f4044c69, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.356362 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7fd3ac-b186-46b0-9214-b282e80e754b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.415610 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 862/1693] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:33 2022] PUT /identity/v3/projects/82ce5689ca214792a7ab40f4905c47f6/users/b29cf33afd1443b68b49d5b5f4044c69/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:33.419501 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0459cd1b-4247-4de7-975b-652a23824bc2 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.435925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0459cd1b-4247-4de7-975b-652a23824bc2 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.479843 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7fd3ac-b186-46b0-9214-b282e80e754b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.585644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8246ce6-b6c1-4d78-9bbe-9d801c89e00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.586293 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8246ce6-b6c1-4d78-9bbe-9d801c89e00f admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.587519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8246ce6-b6c1-4d78-9bbe-9d801c89e00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.587829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8246ce6-b6c1-4d78-9bbe-9d801c89e00f admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.590280 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8246ce6-b6c1-4d78-9bbe-9d801c89e00f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.592684 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8246ce6-b6c1-4d78-9bbe-9d801c89e00f admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.613961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7fd3ac-b186-46b0-9214-b282e80e754b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.622188 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 832/1694] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:33.627154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8246ce6-b6c1-4d78-9bbe-9d801c89e00f admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.627804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378301e-b5be-46e5-8814-62750a0ce661 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.628450 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378301e-b5be-46e5-8814-62750a0ce661 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.628732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378301e-b5be-46e5-8814-62750a0ce661 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.629006 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378301e-b5be-46e5-8814-62750a0ce661 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.648692 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8246ce6-b6c1-4d78-9bbe-9d801c89e00f admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.656556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac8bcf-0fbe-4c40-8876-0f80128e8c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:33.657768 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac8bcf-0fbe-4c40-8876-0f80128e8c48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.658064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac8bcf-0fbe-4c40-8876-0f80128e8c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.658339 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac8bcf-0fbe-4c40-8876-0f80128e8c48 admin admin] PATH_INFO: `/v3/projects/82ce5689ca214792a7ab40f4905c47f6/users/b29cf33afd1443b68b49d5b5f4044c69/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.660842 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac8bcf-0fbe-4c40-8876-0f80128e8c48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82ce5689ca214792a7ab40f4905c47f6, user_id=b29cf33afd1443b68b49d5b5f4044c69, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.664183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac8bcf-0fbe-4c40-8876-0f80128e8c48 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.680738 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 833/1695] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:33 2022] PUT /identity/v3/projects/82ce5689ca214792a7ab40f4905c47f6/users/b29cf33afd1443b68b49d5b5f4044c69/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:33.696571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86442ebb-4c5e-4c98-be4f-2006fe3852d5 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:33.696714 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86442ebb-4c5e-4c98-be4f-2006fe3852d5 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:33.697244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86442ebb-4c5e-4c98-be4f-2006fe3852d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:33.722271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:33.776286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-86442ebb-4c5e-4c98-be4f-2006fe3852d5 None None] MFA Rules not processed for user `b29cf33afd1443b68b49d5b5f4044c69`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:33.783522 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-86442ebb-4c5e-4c98-be4f-2006fe3852d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:33.797994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378301e-b5be-46e5-8814-62750a0ce661 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:33.803067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378301e-b5be-46e5-8814-62750a0ce661 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:33.845284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 863/1696] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:33.846595 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378301e-b5be-46e5-8814-62750a0ce661 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.848281 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:33.855865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3378301e-b5be-46e5-8814-62750a0ce661 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:33.919284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378301e-b5be-46e5-8814-62750a0ce661 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:33.979870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 834/1697] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:33.992278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86442ebb-4c5e-4c98-be4f-2006fe3852d5 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:33.993808 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-86442ebb-4c5e-4c98-be4f-2006fe3852d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:34.118757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.119596 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.120058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.120443 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68 admin admin] PATH_INFO: `/v3/projects/e35f67b84ccf4dcf9b239933a624c61d` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.123829 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e35f67b84ccf4dcf9b239933a624c61d)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.127593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.130565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86442ebb-4c5e-4c98-be4f-2006fe3852d5 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.140504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17068e-6886-45ef-ba13-81596e080117 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.143426 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17068e-6886-45ef-ba13-81596e080117 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.143803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17068e-6886-45ef-ba13-81596e080117 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.144092 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17068e-6886-45ef-ba13-81596e080117 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.176742 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17068e-6886-45ef-ba13-81596e080117 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.179067 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17068e-6886-45ef-ba13-81596e080117 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.208450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 835/1698] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:32:34.314762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e35f67b84ccf4dcf9b239933a624c61d', 'request_id': 'req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:34.362055 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e35f67b84ccf4dcf9b239933a624c61d', 'request_id': 'req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:34.381485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e35f67b84ccf4dcf9b239933a624c61d', 'request_id': 'req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:34.402764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68 admin admin] The token cache is being invalidate because project e35f67b84ccf4dcf9b239933a624c61d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:34.404126 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:34.425089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:34.430748 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 864/1699] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:33 2022] DELETE /identity/v3/projects/e35f67b84ccf4dcf9b239933a624c61d => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:34.435604 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17068e-6886-45ef-ba13-81596e080117 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:34.455541 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17068e-6886-45ef-ba13-81596e080117 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:34.648621 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fa420-141a-434d-85b3-0e147a02e21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:34.649550 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fa420-141a-434d-85b3-0e147a02e21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:34.649925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fa420-141a-434d-85b3-0e147a02e21e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:34.650296 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fa420-141a-434d-85b3-0e147a02e21e admin admin] PATH_INFO: `/v3/users/2061826a8a614ddeb070c31f54211fd9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:34.654015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8fa420-141a-434d-85b3-0e147a02e21e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2061826a8a614ddeb070c31f54211fd9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:34.656031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8fa420-141a-434d-85b3-0e147a02e21e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:34.830545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-0c8fa420-141a-434d-85b3-0e147a02e21e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2061826a8a614ddeb070c31f54211fd9'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:34.831957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-0c8fa420-141a-434d-85b3-0e147a02e21e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2061826a8a614ddeb070c31f54211fd9'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:34.834662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-0c8fa420-141a-434d-85b3-0e147a02e21e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2061826a8a614ddeb070c31f54211fd9'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:34.857799 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-0c8fa420-141a-434d-85b3-0e147a02e21e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2061826a8a614ddeb070c31f54211fd9'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:34.907214 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-0c8fa420-141a-434d-85b3-0e147a02e21e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2061826a8a614ddeb070c31f54211fd9'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:34.916091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:34.985127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 836/1700] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:34 2022] DELETE /identity/v3/users/2061826a8a614ddeb070c31f54211fd9 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:35.164328 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.165693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:35.195116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccf383-97fa-43a9-8ec2-c0d704c1ef68 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:35.369545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599d758-24f6-4a5d-bdc1-230bc07c5cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:35.371227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599d758-24f6-4a5d-bdc1-230bc07c5cea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.372645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599d758-24f6-4a5d-bdc1-230bc07c5cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.373386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599d758-24f6-4a5d-bdc1-230bc07c5cea admin admin] PATH_INFO: `/v3/users/840b4225d8304fd1919cdeafb15cf901` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.376634 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599d758-24f6-4a5d-bdc1-230bc07c5cea admin admin] RBAC: Authorizing `identity:delete_user(user_id=840b4225d8304fd1919cdeafb15cf901)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:35.381949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599d758-24f6-4a5d-bdc1-230bc07c5cea admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:35.407360 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:35.521529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b599d758-24f6-4a5d-bdc1-230bc07c5cea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '840b4225d8304fd1919cdeafb15cf901'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:35.522613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b599d758-24f6-4a5d-bdc1-230bc07c5cea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '840b4225d8304fd1919cdeafb15cf901'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:35.537925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b599d758-24f6-4a5d-bdc1-230bc07c5cea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '840b4225d8304fd1919cdeafb15cf901'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:35.559830 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b599d758-24f6-4a5d-bdc1-230bc07c5cea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '840b4225d8304fd1919cdeafb15cf901'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:35.588889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b599d758-24f6-4a5d-bdc1-230bc07c5cea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '840b4225d8304fd1919cdeafb15cf901'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:35.609390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 865/1701] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:35 2022] DELETE /identity/v3/users/840b4225d8304fd1919cdeafb15cf901 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:35.610904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede5ee7-6935-427d-909d-6e53b13c6c0b None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:35.611385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede5ee7-6935-427d-909d-6e53b13c6c0b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:35.611673 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede5ee7-6935-427d-909d-6e53b13c6c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:35.667883 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-4ede5ee7-6935-427d-909d-6e53b13c6c0b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:35.669599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-4ede5ee7-6935-427d-909d-6e53b13c6c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:35.797934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 837/1702] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:35.919594 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599d758-24f6-4a5d-bdc1-230bc07c5cea admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:35.923195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b599d758-24f6-4a5d-bdc1-230bc07c5cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:36.116584 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599d758-24f6-4a5d-bdc1-230bc07c5cea admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:36.120830 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e7bcb-1956-40b4-9f75-60d59b6aaae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:36.121492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e7bcb-1956-40b4-9f75-60d59b6aaae3 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.121778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e7bcb-1956-40b4-9f75-60d59b6aaae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.122509 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e7bcb-1956-40b4-9f75-60d59b6aaae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.134697 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e7bcb-1956-40b4-9f75-60d59b6aaae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:36.136536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e7bcb-1956-40b4-9f75-60d59b6aaae3 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:36.162384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 866/1703] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:36.911024 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f475e38-dc88-4f2a-ac57-1d61e84f5024 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:36.911535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f475e38-dc88-4f2a-ac57-1d61e84f5024 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:36.912025 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f475e38-dc88-4f2a-ac57-1d61e84f5024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:36.996720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-2f475e38-dc88-4f2a-ac57-1d61e84f5024 None None] MFA Rules not processed for user `28dac3ee2d5c4ff5ba2d15b59e57d2a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:36.998132 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2f475e38-dc88-4f2a-ac57-1d61e84f5024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:37.206614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 838/1704] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:37.226446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e7bcb-1956-40b4-9f75-60d59b6aaae3 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.227930 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bd4e7bcb-1956-40b4-9f75-60d59b6aaae3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:37.287808 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb721c1-dcaa-4d6f-87ac-ebacd5422e59 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.288137 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb721c1-dcaa-4d6f-87ac-ebacd5422e59 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.288365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb721c1-dcaa-4d6f-87ac-ebacd5422e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.370115 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-beb721c1-dcaa-4d6f-87ac-ebacd5422e59 None None] MFA Rules not processed for user `ed0d97f3f08f46af851492aee6a8d0ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:37.371749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-beb721c1-dcaa-4d6f-87ac-ebacd5422e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:37.426451 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e7bcb-1956-40b4-9f75-60d59b6aaae3 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.433286 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe03026-2624-47a9-ab02-53b33ce65d81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.433502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe03026-2624-47a9-ab02-53b33ce65d81 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.433745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe03026-2624-47a9-ab02-53b33ce65d81 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.434021 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe03026-2624-47a9-ab02-53b33ce65d81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.475689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe03026-2624-47a9-ab02-53b33ce65d81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.478069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe03026-2624-47a9-ab02-53b33ce65d81 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.503452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 867/1705] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:37 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:37.504915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d0cfd-bdd2-46e3-8ae8-05fc157c1220 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.505223 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d0cfd-bdd2-46e3-8ae8-05fc157c1220 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.505493 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d0cfd-bdd2-46e3-8ae8-05fc157c1220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.554821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:37.571968 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-eb1d0cfd-bdd2-46e3-8ae8-05fc157c1220 None None] MFA Rules not processed for user `539573490e0d4e128caf93c51567ed79`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:37.573338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-eb1d0cfd-bdd2-46e3-8ae8-05fc157c1220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:37.603240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 839/1706] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:37.609651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb721c1-dcaa-4d6f-87ac-ebacd5422e59 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.615058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-beb721c1-dcaa-4d6f-87ac-ebacd5422e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:37.633767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:37.666679 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb721c1-dcaa-4d6f-87ac-ebacd5422e59 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.693229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6afc5-bd2a-4d9f-a573-969283832f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.695057 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6afc5-bd2a-4d9f-a573-969283832f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.695353 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6afc5-bd2a-4d9f-a573-969283832f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.695644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6afc5-bd2a-4d9f-a573-969283832f4b admin admin] PATH_INFO: `/v3/projects/3d86d0b1ee1c4bea84c6511ef8c57d33` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.709760 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6afc5-bd2a-4d9f-a573-969283832f4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d86d0b1ee1c4bea84c6511ef8c57d33)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.723834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6afc5-bd2a-4d9f-a573-969283832f4b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.736684 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 868/1707] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:37.739965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d0cfd-bdd2-46e3-8ae8-05fc157c1220 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:37.741871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-eb1d0cfd-bdd2-46e3-8ae8-05fc157c1220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:37.782283 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d0cfd-bdd2-46e3-8ae8-05fc157c1220 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:37.798625 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a4f29-9496-40aa-b91d-d2ce39498520 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:37.799321 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a4f29-9496-40aa-b91d-d2ce39498520 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.800157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a4f29-9496-40aa-b91d-d2ce39498520 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.800450 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a4f29-9496-40aa-b91d-d2ce39498520 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.821949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a4f29-9496-40aa-b91d-d2ce39498520 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:37.823850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a4f29-9496-40aa-b91d-d2ce39498520 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:37.847343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 869/1708] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:37.857316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb4099-8411-4685-b9d0-a3fb29f2a080 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:37.857672 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb4099-8411-4685-b9d0-a3fb29f2a080 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:37.858558 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb4099-8411-4685-b9d0-a3fb29f2a080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:37.918722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c3cb4099-8411-4685-b9d0-a3fb29f2a080 None None] MFA Rules not processed for user `539573490e0d4e128caf93c51567ed79`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:37.921751 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b9a6afc5-bd2a-4d9f-a573-969283832f4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d86d0b1ee1c4bea84c6511ef8c57d33', 'request_id': 'req-b9a6afc5-bd2a-4d9f-a573-969283832f4b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:37.928339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c3cb4099-8411-4685-b9d0-a3fb29f2a080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:37.937516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b9a6afc5-bd2a-4d9f-a573-969283832f4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d86d0b1ee1c4bea84c6511ef8c57d33', 'request_id': 'req-b9a6afc5-bd2a-4d9f-a573-969283832f4b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:37.966282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b9a6afc5-bd2a-4d9f-a573-969283832f4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d86d0b1ee1c4bea84c6511ef8c57d33', 'request_id': 'req-b9a6afc5-bd2a-4d9f-a573-969283832f4b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:37.989865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b9a6afc5-bd2a-4d9f-a573-969283832f4b admin admin] The token cache is being invalidate because project 3d86d0b1ee1c4bea84c6511ef8c57d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:37.990552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b9a6afc5-bd2a-4d9f-a573-969283832f4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:37.992710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b9a6afc5-bd2a-4d9f-a573-969283832f4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:38.007635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 840/1709] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:37 2022] DELETE /identity/v3/projects/3d86d0b1ee1c4bea84c6511ef8c57d33 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:38.009990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6afc5-bd2a-4d9f-a573-969283832f4b admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:38.013415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b9a6afc5-bd2a-4d9f-a573-969283832f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:38.109531 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 870/1710] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:38.124609 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb4099-8411-4685-b9d0-a3fb29f2a080 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:38.126461 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c3cb4099-8411-4685-b9d0-a3fb29f2a080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:38.305171 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6afc5-bd2a-4d9f-a573-969283832f4b admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:38.332928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0dd99a-6bb6-47b5-8db2-b1bce9428255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:38.334583 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0dd99a-6bb6-47b5-8db2-b1bce9428255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:38.334883 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0dd99a-6bb6-47b5-8db2-b1bce9428255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:38.335611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0dd99a-6bb6-47b5-8db2-b1bce9428255 admin admin] PATH_INFO: `/v3/users/b29cf33afd1443b68b49d5b5f4044c69` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:38.339088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0dd99a-6bb6-47b5-8db2-b1bce9428255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b29cf33afd1443b68b49d5b5f4044c69)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:38.345100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0dd99a-6bb6-47b5-8db2-b1bce9428255 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:38.351570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb4099-8411-4685-b9d0-a3fb29f2a080 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:38.368179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9743c0-7719-40db-afa2-48a4b8c81a74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:38.368823 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9743c0-7719-40db-afa2-48a4b8c81a74 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:38.369087 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9743c0-7719-40db-afa2-48a4b8c81a74 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:38.370600 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9743c0-7719-40db-afa2-48a4b8c81a74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:38.388173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4a9743c0-7719-40db-afa2-48a4b8c81a74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:38.417632 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9743c0-7719-40db-afa2-48a4b8c81a74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:38.419293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9743c0-7719-40db-afa2-48a4b8c81a74 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:38.448769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 871/1711] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:38.455389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9743c0-7719-40db-afa2-48a4b8c81a74 service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:38.456999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4a9743c0-7719-40db-afa2-48a4b8c81a74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:38.495653 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6c0dd99a-6bb6-47b5-8db2-b1bce9428255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b29cf33afd1443b68b49d5b5f4044c69'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:38.496377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6c0dd99a-6bb6-47b5-8db2-b1bce9428255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b29cf33afd1443b68b49d5b5f4044c69'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:38.496992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6c0dd99a-6bb6-47b5-8db2-b1bce9428255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b29cf33afd1443b68b49d5b5f4044c69'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:38.509103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9743c0-7719-40db-afa2-48a4b8c81a74 service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:38.520427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6c0dd99a-6bb6-47b5-8db2-b1bce9428255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b29cf33afd1443b68b49d5b5f4044c69'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:38.529137 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89c4aa-e19c-479d-8426-5b349cb20e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:38.529825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89c4aa-e19c-479d-8426-5b349cb20e05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:38.530258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89c4aa-e19c-479d-8426-5b349cb20e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:38.530734 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89c4aa-e19c-479d-8426-5b349cb20e05 admin admin] PATH_INFO: `/v3/projects/fa767e5b2ae34a91943e9449dff5476f` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:38.538481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89c4aa-e19c-479d-8426-5b349cb20e05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa767e5b2ae34a91943e9449dff5476f)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:38.540554 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89c4aa-e19c-479d-8426-5b349cb20e05 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:38.551292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6c0dd99a-6bb6-47b5-8db2-b1bce9428255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b29cf33afd1443b68b49d5b5f4044c69'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:38.596615 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:38.599257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 841/1712] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:38 2022] DELETE /identity/v3/users/b29cf33afd1443b68b49d5b5f4044c69 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:38.789105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fe89c4aa-e19c-479d-8426-5b349cb20e05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa767e5b2ae34a91943e9449dff5476f', 'request_id': 'req-fe89c4aa-e19c-479d-8426-5b349cb20e05'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:38.813923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fe89c4aa-e19c-479d-8426-5b349cb20e05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa767e5b2ae34a91943e9449dff5476f', 'request_id': 'req-fe89c4aa-e19c-479d-8426-5b349cb20e05'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:38.840449 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fe89c4aa-e19c-479d-8426-5b349cb20e05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa767e5b2ae34a91943e9449dff5476f', 'request_id': 'req-fe89c4aa-e19c-479d-8426-5b349cb20e05'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:38.869783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fe89c4aa-e19c-479d-8426-5b349cb20e05 admin admin] The token cache is being invalidate because project fa767e5b2ae34a91943e9449dff5476f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:38.870250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fe89c4aa-e19c-479d-8426-5b349cb20e05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:38.871102 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fe89c4aa-e19c-479d-8426-5b349cb20e05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:38.872847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 872/1713] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:38 2022] DELETE /identity/v3/projects/fa767e5b2ae34a91943e9449dff5476f => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:38.878554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0dd99a-6bb6-47b5-8db2-b1bce9428255 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:38.880349 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-6c0dd99a-6bb6-47b5-8db2-b1bce9428255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:38.890803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:39.054594 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0dd99a-6bb6-47b5-8db2-b1bce9428255 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:39.154605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5868e-7ce5-45d9-bbdd-8ad7f1e07b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:39.155380 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5868e-7ce5-45d9-bbdd-8ad7f1e07b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:39.155786 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5868e-7ce5-45d9-bbdd-8ad7f1e07b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:39.156067 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5868e-7ce5-45d9-bbdd-8ad7f1e07b71 admin admin] PATH_INFO: `/v3/users/afc0628e8883482aa6ef95cf10ab8b8d` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:39.158310 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5868e-7ce5-45d9-bbdd-8ad7f1e07b71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afc0628e8883482aa6ef95cf10ab8b8d)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:39.160767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5868e-7ce5-45d9-bbdd-8ad7f1e07b71 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:39.247863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d5e5868e-7ce5-45d9-bbdd-8ad7f1e07b71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afc0628e8883482aa6ef95cf10ab8b8d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:39.248988 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d5e5868e-7ce5-45d9-bbdd-8ad7f1e07b71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afc0628e8883482aa6ef95cf10ab8b8d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:39.249619 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d5e5868e-7ce5-45d9-bbdd-8ad7f1e07b71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afc0628e8883482aa6ef95cf10ab8b8d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:39.262933 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d5e5868e-7ce5-45d9-bbdd-8ad7f1e07b71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afc0628e8883482aa6ef95cf10ab8b8d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:39.284469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d5e5868e-7ce5-45d9-bbdd-8ad7f1e07b71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afc0628e8883482aa6ef95cf10ab8b8d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:39.328808 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 842/1714] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:38 2022] DELETE /identity/v3/users/afc0628e8883482aa6ef95cf10ab8b8d => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:39.643178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14bf8e-cc19-44f7-9d43-dd171e11935f None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:39.643666 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14bf8e-cc19-44f7-9d43-dd171e11935f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:39.643807 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14bf8e-cc19-44f7-9d43-dd171e11935f None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:39.645540 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 873/1715] 10.222.0.58 () {54 vars in 912 bytes} [Wed Aug 24 12:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:32:39.651866 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298795-cad0-4724-8b2e-9711956dd3bb None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:39.652155 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298795-cad0-4724-8b2e-9711956dd3bb None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:39.652534 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68298795-cad0-4724-8b2e-9711956dd3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:39.738788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-68298795-cad0-4724-8b2e-9711956dd3bb None None] MFA Rules not processed for user `34fcb5885325472bbe338cf05d3788f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:39.740254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-68298795-cad0-4724-8b2e-9711956dd3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:39.773156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:39.854733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 843/1716] 10.222.0.58 () {58 vars in 1009 bytes} [Wed Aug 24 12:32:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:40.051141 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14bf8e-cc19-44f7-9d43-dd171e11935f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:40.053672 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6c14bf8e-cc19-44f7-9d43-dd171e11935f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:40.084394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14bf8e-cc19-44f7-9d43-dd171e11935f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:40.100528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef7383-2951-4574-91a4-9d29fba91164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:40.101241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef7383-2951-4574-91a4-9d29fba91164 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:40.101532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef7383-2951-4574-91a4-9d29fba91164 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:40.101813 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef7383-2951-4574-91a4-9d29fba91164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:40.111291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef7383-2951-4574-91a4-9d29fba91164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:40.117931 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef7383-2951-4574-91a4-9d29fba91164 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:40.142551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 874/1717] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:40.387744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298795-cad0-4724-8b2e-9711956dd3bb None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:40.389521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-68298795-cad0-4724-8b2e-9711956dd3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:40.554105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68298795-cad0-4724-8b2e-9711956dd3bb None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:40.558920 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af156c-4ea7-4122-88f0-b17049d320f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:40.559665 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af156c-4ea7-4122-88f0-b17049d320f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:40.559944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af156c-4ea7-4122-88f0-b17049d320f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:40.560223 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af156c-4ea7-4122-88f0-b17049d320f7 admin admin] PATH_INFO: `/v3/projects/82ce5689ca214792a7ab40f4905c47f6` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:40.561947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af156c-4ea7-4122-88f0-b17049d320f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82ce5689ca214792a7ab40f4905c47f6)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:40.564379 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af156c-4ea7-4122-88f0-b17049d320f7 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:40.662737 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-73af156c-4ea7-4122-88f0-b17049d320f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82ce5689ca214792a7ab40f4905c47f6', 'request_id': 'req-73af156c-4ea7-4122-88f0-b17049d320f7'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:40.671944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-73af156c-4ea7-4122-88f0-b17049d320f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82ce5689ca214792a7ab40f4905c47f6', 'request_id': 'req-73af156c-4ea7-4122-88f0-b17049d320f7'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:40.688891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-73af156c-4ea7-4122-88f0-b17049d320f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82ce5689ca214792a7ab40f4905c47f6', 'request_id': 'req-73af156c-4ea7-4122-88f0-b17049d320f7'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:40.704793 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-73af156c-4ea7-4122-88f0-b17049d320f7 admin admin] The token cache is being invalidate because project 82ce5689ca214792a7ab40f4905c47f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:40.705021 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-73af156c-4ea7-4122-88f0-b17049d320f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:40.706407 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-73af156c-4ea7-4122-88f0-b17049d320f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:40.714895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 844/1718] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:40 2022] DELETE /identity/v3/projects/82ce5689ca214792a7ab40f4905c47f6 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:40.720501 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912492a1-fb2e-4608-91ff-d8307ad36ad9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:40.721315 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912492a1-fb2e-4608-91ff-d8307ad36ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:40.721315 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912492a1-fb2e-4608-91ff-d8307ad36ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:40.777993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-912492a1-fb2e-4608-91ff-d8307ad36ad9 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:40.779520 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-912492a1-fb2e-4608-91ff-d8307ad36ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:40.920380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 875/1719] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:40.926784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750ba51-3b0a-4f36-8d25-54fb635ad8bc None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:40.927072 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750ba51-3b0a-4f36-8d25-54fb635ad8bc None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:40.927346 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a750ba51-3b0a-4f36-8d25-54fb635ad8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:40.972168 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-a750ba51-3b0a-4f36-8d25-54fb635ad8bc None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:40.976590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a750ba51-3b0a-4f36-8d25-54fb635ad8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:40.999731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 845/1720] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:41.013047 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912492a1-fb2e-4608-91ff-d8307ad36ad9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:41.036212 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912492a1-fb2e-4608-91ff-d8307ad36ad9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:41.047600 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970eb632-8735-4acb-a369-73f6795af26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:41.048974 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970eb632-8735-4acb-a369-73f6795af26c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:41.053611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970eb632-8735-4acb-a369-73f6795af26c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:41.055387 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970eb632-8735-4acb-a369-73f6795af26c admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:41.056380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970eb632-8735-4acb-a369-73f6795af26c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:41.057723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970eb632-8735-4acb-a369-73f6795af26c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:41.060195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970eb632-8735-4acb-a369-73f6795af26c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:41.099738 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 876/1721] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:41.104528 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750ba51-3b0a-4f36-8d25-54fb635ad8bc None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:41.121711 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a750ba51-3b0a-4f36-8d25-54fb635ad8bc None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:41.138511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fa10b-52e8-4539-a1f4-78628040c27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:41.142365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fa10b-52e8-4539-a1f4-78628040c27c admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:41.142660 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fa10b-52e8-4539-a1f4-78628040c27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:41.142948 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fa10b-52e8-4539-a1f4-78628040c27c admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:41.147088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9fa10b-52e8-4539-a1f4-78628040c27c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:41.149084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9fa10b-52e8-4539-a1f4-78628040c27c admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:41.240088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 846/1722] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:41 2022] POST /identity/v3/projects => generated 363 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:41.245974 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970eb632-8735-4acb-a369-73f6795af26c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:41.268224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970eb632-8735-4acb-a369-73f6795af26c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:41.488662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79bfaa-624b-4d40-a546-19eb3a452f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:41.489716 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79bfaa-624b-4d40-a546-19eb3a452f59 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:41.490168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79bfaa-624b-4d40-a546-19eb3a452f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:41.490670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79bfaa-624b-4d40-a546-19eb3a452f59 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:41.491953 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79bfaa-624b-4d40-a546-19eb3a452f59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:41.494983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79bfaa-624b-4d40-a546-19eb3a452f59 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:41.628046 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fa10b-52e8-4539-a1f4-78628040c27c admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:41.633941 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 877/1723] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:41 2022] POST /identity/v3/users => generated 282 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:41.664727 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fa10b-52e8-4539-a1f4-78628040c27c admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:41.703116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3c938-81cc-4621-ac64-28ef84acf4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:41.703854 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3c938-81cc-4621-ac64-28ef84acf4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:41.704135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3c938-81cc-4621-ac64-28ef84acf4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:41.733851 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3c938-81cc-4621-ac64-28ef84acf4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:41.735007 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3c938-81cc-4621-ac64-28ef84acf4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:41.743134 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3c938-81cc-4621-ac64-28ef84acf4b5 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:41.829562 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 847/1724] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:41.835004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79bfaa-624b-4d40-a546-19eb3a452f59 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:41.844433 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:41.851760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79bfaa-624b-4d40-a546-19eb3a452f59 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:41.870980 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc936f-5a78-4a23-b0f9-1f0e069d5030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:41.872011 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc936f-5a78-4a23-b0f9-1f0e069d5030 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:41.872451 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc936f-5a78-4a23-b0f9-1f0e069d5030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:41.872882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc936f-5a78-4a23-b0f9-1f0e069d5030 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:41.874398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc936f-5a78-4a23-b0f9-1f0e069d5030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:41.887522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bc936f-5a78-4a23-b0f9-1f0e069d5030 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:41.933992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 878/1725] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:41.938592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3c938-81cc-4621-ac64-28ef84acf4b5 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:41.963428 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3c938-81cc-4621-ac64-28ef84acf4b5 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:41.984769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4ed24-c1d1-44af-802d-5add109bc1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:41.989198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4ed24-c1d1-44af-802d-5add109bc1de admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:41.989198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4ed24-c1d1-44af-802d-5add109bc1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:41.989198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4ed24-c1d1-44af-802d-5add109bc1de admin admin] PATH_INFO: `/v3/projects/98bc9d2fce244b0890a847f3ef9e4169/users/1df46d574e5641389fe6372bcbd5f51f/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:42.042126 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a4ed24-c1d1-44af-802d-5add109bc1de admin admin] RBAC: Authorizing `identity:create_grant(project_id=98bc9d2fce244b0890a847f3ef9e4169, user_id=1df46d574e5641389fe6372bcbd5f51f, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:42.045858 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a4ed24-c1d1-44af-802d-5add109bc1de admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:42.065976 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:42.099528 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 848/1726] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:41 2022] PUT /identity/v3/projects/98bc9d2fce244b0890a847f3ef9e4169/users/1df46d574e5641389fe6372bcbd5f51f/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:42.115512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551e047-884f-4938-92f9-ad15f2f6b6b6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:42.116506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551e047-884f-4938-92f9-ad15f2f6b6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:42.117047 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551e047-884f-4938-92f9-ad15f2f6b6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:42.208801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3551e047-884f-4938-92f9-ad15f2f6b6b6 None None] MFA Rules not processed for user `1df46d574e5641389fe6372bcbd5f51f`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:42.213668 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3551e047-884f-4938-92f9-ad15f2f6b6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:42.493984 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 879/1727] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:42.500660 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4ed24-c1d1-44af-802d-5add109bc1de admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:42.511679 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e93d8-a1a1-4a83-b53c-8c8214e28929 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:42.512074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e93d8-a1a1-4a83-b53c-8c8214e28929 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:42.512411 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e93d8-a1a1-4a83-b53c-8c8214e28929 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:42.514506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 880/1728] 10.222.0.58 () {54 vars in 961 bytes} [Wed Aug 24 12:32:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:32:42.537388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd3bc1-3d78-4002-bec4-e18aa94325aa None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:42.538639 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd3bc1-3d78-4002-bec4-e18aa94325aa None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:42.538989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd3bc1-3d78-4002-bec4-e18aa94325aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:42.582447 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f5fd3bc1-3d78-4002-bec4-e18aa94325aa None None] MFA Rules not processed for user `b7ba45e3cff74962965b248f9f017f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:42.583943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f5fd3bc1-3d78-4002-bec4-e18aa94325aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:42.721561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4ed24-c1d1-44af-802d-5add109bc1de admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:42.735843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513339d5-3c5d-43a7-920f-3c8a5c7899dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:42.736665 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513339d5-3c5d-43a7-920f-3c8a5c7899dd admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:42.737059 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513339d5-3c5d-43a7-920f-3c8a5c7899dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:42.737447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513339d5-3c5d-43a7-920f-3c8a5c7899dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:42.742313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 881/1729] 10.222.0.58 () {58 vars in 1058 bytes} [Wed Aug 24 12:32:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:42.755580 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513339d5-3c5d-43a7-920f-3c8a5c7899dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:42.756879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd3bc1-3d78-4002-bec4-e18aa94325aa None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:42.758676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513339d5-3c5d-43a7-920f-3c8a5c7899dd admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:42.782997 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd3bc1-3d78-4002-bec4-e18aa94325aa None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:42.794854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e82e8-a639-443f-be0b-ad4cb7687c25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ba45e3cff74962965b248f9f017f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:42.797595 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e82e8-a639-443f-be0b-ad4cb7687c25 service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:42.799486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e82e8-a639-443f-be0b-ad4cb7687c25 service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:42.801061 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e82e8-a639-443f-be0b-ad4cb7687c25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:42.805332 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b42e82e8-a639-443f-be0b-ad4cb7687c25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:42.833309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 849/1730] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:42 2022] POST /identity/v3/projects => generated 363 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:42.837431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513339d5-3c5d-43a7-920f-3c8a5c7899dd admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:42.849638 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e82e8-a639-443f-be0b-ad4cb7687c25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:42.855177 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42e82e8-a639-443f-be0b-ad4cb7687c25 service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:42.914656 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 882/1731] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:43.063389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513339d5-3c5d-43a7-920f-3c8a5c7899dd admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:43.081038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59745f-14da-48da-9321-b5c8d0c7cc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:43.081819 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59745f-14da-48da-9321-b5c8d0c7cc41 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:43.082100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59745f-14da-48da-9321-b5c8d0c7cc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:43.082373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59745f-14da-48da-9321-b5c8d0c7cc41 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:43.083011 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59745f-14da-48da-9321-b5c8d0c7cc41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:43.091716 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c59745f-14da-48da-9321-b5c8d0c7cc41 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:43.150571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 850/1732] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:42 2022] POST /identity/v3/users => generated 288 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:32:43.155386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e82e8-a639-443f-be0b-ad4cb7687c25 service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:43.180416 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42e82e8-a639-443f-be0b-ad4cb7687c25 service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:43.205777 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d5a09-0a3d-47f6-8941-3d183566fb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:43.213618 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d5a09-0a3d-47f6-8941-3d183566fb8f admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:43.214110 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d5a09-0a3d-47f6-8941-3d183566fb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:43.214394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d5a09-0a3d-47f6-8941-3d183566fb8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:43.222204 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d5a09-0a3d-47f6-8941-3d183566fb8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:43.226447 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d5a09-0a3d-47f6-8941-3d183566fb8f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:43.295719 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 883/1733] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:43.298107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59745f-14da-48da-9321-b5c8d0c7cc41 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:43.317502 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59745f-14da-48da-9321-b5c8d0c7cc41 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:43.343753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e78ba-ff7d-402e-a290-d0d026f2effa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:43.344498 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e78ba-ff7d-402e-a290-d0d026f2effa admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:43.344787 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e78ba-ff7d-402e-a290-d0d026f2effa admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:43.345100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e78ba-ff7d-402e-a290-d0d026f2effa admin admin] PATH_INFO: `/v3/projects/c0a1fcc10cf44e0eb74b20e0afb9a0d0/users/ee32af7b9a1243e3b8258a67d161c8bc/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:43.414536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e78ba-ff7d-402e-a290-d0d026f2effa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0a1fcc10cf44e0eb74b20e0afb9a0d0, user_id=ee32af7b9a1243e3b8258a67d161c8bc, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:43.423193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e78ba-ff7d-402e-a290-d0d026f2effa admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:43.489974 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 851/1734] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:43 2022] PUT /identity/v3/projects/c0a1fcc10cf44e0eb74b20e0afb9a0d0/users/ee32af7b9a1243e3b8258a67d161c8bc/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:43.494432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d5a09-0a3d-47f6-8941-3d183566fb8f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:43.521063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d5a09-0a3d-47f6-8941-3d183566fb8f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:43.771002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:43.795697 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223c2d1-9170-4fe8-bf67-58991066e5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:43.796345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223c2d1-9170-4fe8-bf67-58991066e5d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:43.796630 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223c2d1-9170-4fe8-bf67-58991066e5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:43.796907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223c2d1-9170-4fe8-bf67-58991066e5d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:43.797560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223c2d1-9170-4fe8-bf67-58991066e5d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:43.799203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223c2d1-9170-4fe8-bf67-58991066e5d4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:43.818756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e78ba-ff7d-402e-a290-d0d026f2effa admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:43.821323 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-492e78ba-ff7d-402e-a290-d0d026f2effa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:43.851924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e78ba-ff7d-402e-a290-d0d026f2effa admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:43.862362 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 884/1735] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:43.866369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223c2d1-9170-4fe8-bf67-58991066e5d4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:43.877972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e42b60-061d-42bd-a43f-64fabfb860e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:43.879053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e42b60-061d-42bd-a43f-64fabfb860e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:43.879810 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e42b60-061d-42bd-a43f-64fabfb860e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:43.882093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e42b60-061d-42bd-a43f-64fabfb860e8 admin admin] PATH_INFO: `/v3/projects/9a29ac3f5f60422abf5f2cc4ab09d151` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:43.885126 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e42b60-061d-42bd-a43f-64fabfb860e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a29ac3f5f60422abf5f2cc4ab09d151)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:43.896320 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e42b60-061d-42bd-a43f-64fabfb860e8 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:43.908579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223c2d1-9170-4fe8-bf67-58991066e5d4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:43.936410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57534f3b-67bb-4bf3-9b09-27f1149ab692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:43.940536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57534f3b-67bb-4bf3-9b09-27f1149ab692 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:43.942539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57534f3b-67bb-4bf3-9b09-27f1149ab692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:43.946430 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57534f3b-67bb-4bf3-9b09-27f1149ab692 admin admin] PATH_INFO: `/v3/projects/c0a1fcc10cf44e0eb74b20e0afb9a0d0/users/ee32af7b9a1243e3b8258a67d161c8bc/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:43.960508 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57534f3b-67bb-4bf3-9b09-27f1149ab692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0a1fcc10cf44e0eb74b20e0afb9a0d0, user_id=ee32af7b9a1243e3b8258a67d161c8bc, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:43.966700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57534f3b-67bb-4bf3-9b09-27f1149ab692 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:44.031487 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 885/1736] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:43 2022] PUT /identity/v3/projects/c0a1fcc10cf44e0eb74b20e0afb9a0d0/users/ee32af7b9a1243e3b8258a67d161c8bc/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:44.047497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c8a36-3855-4048-9583-ee378b6cb360 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:44.050924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c8a36-3855-4048-9583-ee378b6cb360 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:44.050924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c8a36-3855-4048-9583-ee378b6cb360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:44.064002 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:INSERT=6,keystone:SELECT=91,keystone:DELETE=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:44.090462 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b4e42b60-061d-42bd-a43f-64fabfb860e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a29ac3f5f60422abf5f2cc4ab09d151', 'request_id': 'req-b4e42b60-061d-42bd-a43f-64fabfb860e8'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:44.112560 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b4e42b60-061d-42bd-a43f-64fabfb860e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a29ac3f5f60422abf5f2cc4ab09d151', 'request_id': 'req-b4e42b60-061d-42bd-a43f-64fabfb860e8'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:44.133452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b4e42b60-061d-42bd-a43f-64fabfb860e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a29ac3f5f60422abf5f2cc4ab09d151', 'request_id': 'req-b4e42b60-061d-42bd-a43f-64fabfb860e8'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:44.164493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b4e42b60-061d-42bd-a43f-64fabfb860e8 admin admin] The token cache is being invalidate because project 9a29ac3f5f60422abf5f2cc4ab09d151 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:44.165509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b4e42b60-061d-42bd-a43f-64fabfb860e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:44.167887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b4e42b60-061d-42bd-a43f-64fabfb860e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:44.180194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 852/1737] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:43 2022] DELETE /identity/v3/projects/9a29ac3f5f60422abf5f2cc4ab09d151 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:44.188485 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b5d32-50c7-4ead-b1c7-96bc53e4d1d6 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:44.189100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b5d32-50c7-4ead-b1c7-96bc53e4d1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:44.190917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b5d32-50c7-4ead-b1c7-96bc53e4d1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:44.217945 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d81c8a36-3855-4048-9583-ee378b6cb360 None None] MFA Rules not processed for user `ee32af7b9a1243e3b8258a67d161c8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:44.226030 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d81c8a36-3855-4048-9583-ee378b6cb360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:44.278949 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-7a6b5d32-50c7-4ead-b1c7-96bc53e4d1d6 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:44.299595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7a6b5d32-50c7-4ead-b1c7-96bc53e4d1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:44.468397 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 853/1738] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:44.473387 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d25bd-56a7-48dd-aa63-522a9ef52584 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:44.473744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d25bd-56a7-48dd-aa63-522a9ef52584 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:44.474093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d25bd-56a7-48dd-aa63-522a9ef52584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:44.480169 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 886/1739] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:44 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:44.506354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c8a36-3855-4048-9583-ee378b6cb360 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:44.509955 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d81c8a36-3855-4048-9583-ee378b6cb360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:44.519649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-d46d25bd-56a7-48dd-aa63-522a9ef52584 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:44.521260 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d46d25bd-56a7-48dd-aa63-522a9ef52584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:44.540854 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 854/1740] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:44.542023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d25bd-56a7-48dd-aa63-522a9ef52584 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:44.557484 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d25bd-56a7-48dd-aa63-522a9ef52584 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:44.572721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a8e10-8401-4016-8bf0-73053d4d4c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:44.573400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a8e10-8401-4016-8bf0-73053d4d4c6c admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:44.573686 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a8e10-8401-4016-8bf0-73053d4d4c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:44.573954 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a8e10-8401-4016-8bf0-73053d4d4c6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:44.574617 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a8e10-8401-4016-8bf0-73053d4d4c6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:44.574922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a8e10-8401-4016-8bf0-73053d4d4c6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:44.576645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a8e10-8401-4016-8bf0-73053d4d4c6c admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:44.602733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 855/1741] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:32:44.607249 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a8e10-8401-4016-8bf0-73053d4d4c6c admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:44.629781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a8e10-8401-4016-8bf0-73053d4d4c6c admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:44.637768 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc185640-d611-444b-b7f0-5bb878bb8c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:44.638400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc185640-d611-444b-b7f0-5bb878bb8c12 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:44.638688 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc185640-d611-444b-b7f0-5bb878bb8c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:44.639117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc185640-d611-444b-b7f0-5bb878bb8c12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:44.639808 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc185640-d611-444b-b7f0-5bb878bb8c12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:44.641737 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc185640-d611-444b-b7f0-5bb878bb8c12 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:44.728117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 856/1742] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:44 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:44.731066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc185640-d611-444b-b7f0-5bb878bb8c12 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:44.799455 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c8a36-3855-4048-9583-ee378b6cb360 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:44.950010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc185640-d611-444b-b7f0-5bb878bb8c12 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:44.971655 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bf4d4-5cd1-43f8-b3c6-7a4100039c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:44.972665 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bf4d4-5cd1-43f8-b3c6-7a4100039c36 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:44.973627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bf4d4-5cd1-43f8-b3c6-7a4100039c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:44.974121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bf4d4-5cd1-43f8-b3c6-7a4100039c36 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:44.974964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bf4d4-5cd1-43f8-b3c6-7a4100039c36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:44.977145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bf4d4-5cd1-43f8-b3c6-7a4100039c36 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:45.019276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f23cf-c12d-4b34-89e4-6f00a934c709 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:45.020488 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f23cf-c12d-4b34-89e4-6f00a934c709 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:45.021024 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f23cf-c12d-4b34-89e4-6f00a934c709 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:45.021793 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f23cf-c12d-4b34-89e4-6f00a934c709 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:45.047882 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 857/1743] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:44 2022] POST /identity/v3/users => generated 288 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:45.059076 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29729cd7-92d7-445d-b969-d6dcdb080a6a None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:45.060500 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29729cd7-92d7-445d-b969-d6dcdb080a6a None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:45.062575 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29729cd7-92d7-445d-b969-d6dcdb080a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:45.154724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:45.156868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-29729cd7-92d7-445d-b969-d6dcdb080a6a None None] MFA Rules not processed for user `9138ac21bbdb4577a7f43b5ab471d572`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:45.159325 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-29729cd7-92d7-445d-b969-d6dcdb080a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:45.168950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:45.219974 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f23cf-c12d-4b34-89e4-6f00a934c709 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:45.221422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f23cf-c12d-4b34-89e4-6f00a934c709 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:45.234805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 887/1744] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 729 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:45.235780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f23cf-c12d-4b34-89e4-6f00a934c709 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:45.253736 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f23cf-c12d-4b34-89e4-6f00a934c709 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:45.261848 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e794a8-cd95-4a6b-9234-5250a6e6903d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:45.262708 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e794a8-cd95-4a6b-9234-5250a6e6903d admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:45.263099 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e794a8-cd95-4a6b-9234-5250a6e6903d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:45.263491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e794a8-cd95-4a6b-9234-5250a6e6903d admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:45.266076 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e794a8-cd95-4a6b-9234-5250a6e6903d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:45.268550 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e794a8-cd95-4a6b-9234-5250a6e6903d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:45.287583 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 888/1745] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:45.288988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9c770-e5c8-456d-9d84-22b18e9f90b9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:45.289282 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9c770-e5c8-456d-9d84-22b18e9f90b9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:45.289558 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9c770-e5c8-456d-9d84-22b18e9f90b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:45.329408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-49e9c770-e5c8-456d-9d84-22b18e9f90b9 None None] MFA Rules not processed for user `9aa753d1e2c4430498d5b28f0304952f`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:45.330960 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-49e9c770-e5c8-456d-9d84-22b18e9f90b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:45.349781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 858/1746] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:45.360748 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29729cd7-92d7-445d-b969-d6dcdb080a6a None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:45.396376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29729cd7-92d7-445d-b969-d6dcdb080a6a None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:45.414847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b4e76-0577-471f-975e-da8e47a0e2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:45.415542 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b4e76-0577-471f-975e-da8e47a0e2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:45.415934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b4e76-0577-471f-975e-da8e47a0e2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:45.416208 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b4e76-0577-471f-975e-da8e47a0e2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:45.416922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b4e76-0577-471f-975e-da8e47a0e2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:45.418553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b4e76-0577-471f-975e-da8e47a0e2a5 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:45.422179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 889/1747] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:45.424762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c0b20-49c3-4256-a940-59a0623225f7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:45.425062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c0b20-49c3-4256-a940-59a0623225f7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:45.425341 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c0b20-49c3-4256-a940-59a0623225f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:45.455777 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-597c0b20-49c3-4256-a940-59a0623225f7 None None] MFA Rules not processed for user `9138ac21bbdb4577a7f43b5ab471d572`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:45.457368 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-597c0b20-49c3-4256-a940-59a0623225f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:45.462489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 859/1748] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:45.471986 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 890/1749] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:45.476123 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c0b20-49c3-4256-a940-59a0623225f7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:45.476382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b4e76-0577-471f-975e-da8e47a0e2a5 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:45.477490 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-597c0b20-49c3-4256-a940-59a0623225f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:45.502553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b4e76-0577-471f-975e-da8e47a0e2a5 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:45.513485 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7720a-2ce2-4726-a8f4-f9201f5ad39c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:45.514860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7720a-2ce2-4726-a8f4-f9201f5ad39c service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:45.515211 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7720a-2ce2-4726-a8f4-f9201f5ad39c service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:45.515487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7720a-2ce2-4726-a8f4-f9201f5ad39c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:45.521493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f2a7720a-2ce2-4726-a8f4-f9201f5ad39c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:45.652585 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7720a-2ce2-4726-a8f4-f9201f5ad39c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:45.654216 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7720a-2ce2-4726-a8f4-f9201f5ad39c service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:45.683256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 860/1750] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:45.684358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7720a-2ce2-4726-a8f4-f9201f5ad39c service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:45.696101 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c0b20-49c3-4256-a940-59a0623225f7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:45.704066 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bbfd37-44b4-45c2-a8ed-66109c75d4d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:45.704824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbfd37-44b4-45c2-a8ed-66109c75d4d1 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:45.705215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbfd37-44b4-45c2-a8ed-66109c75d4d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:45.705613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bbfd37-44b4-45c2-a8ed-66109c75d4d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:45.710532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7720a-2ce2-4726-a8f4-f9201f5ad39c service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:45.721634 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bbfd37-44b4-45c2-a8ed-66109c75d4d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:45.724326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bbfd37-44b4-45c2-a8ed-66109c75d4d1 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:45.727038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ff6e7-b31c-4d23-bd4f-441355d89325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:45.727783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ff6e7-b31c-4d23-bd4f-441355d89325 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:45.728073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ff6e7-b31c-4d23-bd4f-441355d89325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:45.728350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ff6e7-b31c-4d23-bd4f-441355d89325 admin admin] PATH_INFO: `/v3/projects/b777488b8bf94b5fb8ac61300cea8b37/users/d4026e21eeaf46888d239a01d8275aa9/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:45.744847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 891/1751] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:45.770010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ff6e7-b31c-4d23-bd4f-441355d89325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b777488b8bf94b5fb8ac61300cea8b37, user_id=d4026e21eeaf46888d239a01d8275aa9, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:45.775759 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ff6e7-b31c-4d23-bd4f-441355d89325 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:45.809425 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 861/1752] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:45 2022] PUT /identity/v3/projects/b777488b8bf94b5fb8ac61300cea8b37/users/d4026e21eeaf46888d239a01d8275aa9/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:45.821707 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12652ba-f622-4a3e-a6dc-a590e53ace73 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:45.822256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12652ba-f622-4a3e-a6dc-a590e53ace73 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:45.822556 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12652ba-f622-4a3e-a6dc-a590e53ace73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:45.876738 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ff6e7-b31c-4d23-bd4f-441355d89325 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:45.919512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e12652ba-f622-4a3e-a6dc-a590e53ace73 None None] MFA Rules not processed for user `d4026e21eeaf46888d239a01d8275aa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:45.920911 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e12652ba-f622-4a3e-a6dc-a590e53ace73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:46.012958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:46.246446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 892/1753] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:46.248029 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69130591-4652-4f0d-b4b4-16d4cc02f3dc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:46.248483 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69130591-4652-4f0d-b4b4-16d4cc02f3dc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:46.248769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69130591-4652-4f0d-b4b4-16d4cc02f3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:46.249792 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ff6e7-b31c-4d23-bd4f-441355d89325 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:46.267344 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb729e3e-742c-4499-a132-0dbb5377e5b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:46.270517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb729e3e-742c-4499-a132-0dbb5377e5b0 service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:46.270969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb729e3e-742c-4499-a132-0dbb5377e5b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:46.271524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb729e3e-742c-4499-a132-0dbb5377e5b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:46.281870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:46.300658 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-69130591-4652-4f0d-b4b4-16d4cc02f3dc None None] MFA Rules not processed for user `28dac3ee2d5c4ff5ba2d15b59e57d2a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:46.301895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-69130591-4652-4f0d-b4b4-16d4cc02f3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:46.482142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb729e3e-742c-4499-a132-0dbb5377e5b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:46.486205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb729e3e-742c-4499-a132-0dbb5377e5b0 service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:46.510823 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 893/1754] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:46.513545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69130591-4652-4f0d-b4b4-16d4cc02f3dc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:46.539779 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 862/1755] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:46.773117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69130591-4652-4f0d-b4b4-16d4cc02f3dc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:46.782550 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12787bae-ff2e-49c8-833a-0cb237a9be7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:46.784263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12787bae-ff2e-49c8-833a-0cb237a9be7b service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:46.784915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12787bae-ff2e-49c8-833a-0cb237a9be7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:46.785641 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12787bae-ff2e-49c8-833a-0cb237a9be7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:46.800784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12787bae-ff2e-49c8-833a-0cb237a9be7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:46.802657 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12787bae-ff2e-49c8-833a-0cb237a9be7b service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:46.821891 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 894/1756] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:46.912864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399abeaf-0fbd-4e23-ab78-be2d3f867be7 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:46.913439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399abeaf-0fbd-4e23-ab78-be2d3f867be7 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:46.914454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399abeaf-0fbd-4e23-ab78-be2d3f867be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:47.107248 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-399abeaf-0fbd-4e23-ab78-be2d3f867be7 None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:47.114443 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-399abeaf-0fbd-4e23-ab78-be2d3f867be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:47.358764 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 863/1757] 10.222.0.58 () {58 vars in 1009 bytes} [Wed Aug 24 12:32:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:47.384588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e3bf5-8ef1-4d5d-8d96-9816a34a4bba None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:47.385969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e3bf5-8ef1-4d5d-8d96-9816a34a4bba None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:47.386657 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e3bf5-8ef1-4d5d-8d96-9816a34a4bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:47.444866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-682e3bf5-8ef1-4d5d-8d96-9816a34a4bba None None] MFA Rules not processed for user `ed0d97f3f08f46af851492aee6a8d0ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:47.449238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-682e3bf5-8ef1-4d5d-8d96-9816a34a4bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:47.664126 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399abeaf-0fbd-4e23-ab78-be2d3f867be7 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:47.665534 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-399abeaf-0fbd-4e23-ab78-be2d3f867be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:47.726665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 895/1758] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:48.005139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:48.028099 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399abeaf-0fbd-4e23-ab78-be2d3f867be7 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:48.044430 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8e88a-69ae-4ad5-afba-dc0f0ee6726c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:48.046751 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8e88a-69ae-4ad5-afba-dc0f0ee6726c service swift] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:48.048418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8e88a-69ae-4ad5-afba-dc0f0ee6726c service swift] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:48.050257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8e88a-69ae-4ad5-afba-dc0f0ee6726c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:48.073958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8e88a-69ae-4ad5-afba-dc0f0ee6726c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:48.079910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8e88a-69ae-4ad5-afba-dc0f0ee6726c service swift] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:48.109617 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 864/1759] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:32:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 12:32:48.717178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce6433-8ba8-453f-a53b-71460c0314dc None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:48.718046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce6433-8ba8-453f-a53b-71460c0314dc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:48.718046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce6433-8ba8-453f-a53b-71460c0314dc None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:48.720533 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 896/1760] 10.222.0.58 () {54 vars in 912 bytes} [Wed Aug 24 12:32:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:32:48.734803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef956dbf-1659-4c68-8905-ef93d2188ea4 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:48.736097 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef956dbf-1659-4c68-8905-ef93d2188ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:48.736590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef956dbf-1659-4c68-8905-ef93d2188ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:48.836845 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-ef956dbf-1659-4c68-8905-ef93d2188ea4 None None] MFA Rules not processed for user `34fcb5885325472bbe338cf05d3788f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:48.842533 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-ef956dbf-1659-4c68-8905-ef93d2188ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:49.078664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 865/1761] 10.222.0.58 () {58 vars in 1009 bytes} [Wed Aug 24 12:32:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:49.180384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce6433-8ba8-453f-a53b-71460c0314dc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:49.182619 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d4ce6433-8ba8-453f-a53b-71460c0314dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:49.229569 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce6433-8ba8-453f-a53b-71460c0314dc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:49.257451 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f4289-ebc0-4973-9db4-8642a4e8c139 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:49.260197 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f4289-ebc0-4973-9db4-8642a4e8c139 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:49.267448 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f4289-ebc0-4973-9db4-8642a4e8c139 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:49.270492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f4289-ebc0-4973-9db4-8642a4e8c139 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:49.296932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:49.310032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f4289-ebc0-4973-9db4-8642a4e8c139 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:49.322290 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f4289-ebc0-4973-9db4-8642a4e8c139 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:49.353975 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 897/1762] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:49.375156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef956dbf-1659-4c68-8905-ef93d2188ea4 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:49.376958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-ef956dbf-1659-4c68-8905-ef93d2188ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:49.807658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef956dbf-1659-4c68-8905-ef93d2188ea4 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:49.839560 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5f884-5f8c-4ccd-8b45-6ba2f4dfa55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:49.849107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5f884-5f8c-4ccd-8b45-6ba2f4dfa55e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:49.851389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5f884-5f8c-4ccd-8b45-6ba2f4dfa55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:49.854470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5f884-5f8c-4ccd-8b45-6ba2f4dfa55e admin admin] PATH_INFO: `/v3/users/1df46d574e5641389fe6372bcbd5f51f` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:49.867874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5f884-5f8c-4ccd-8b45-6ba2f4dfa55e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1df46d574e5641389fe6372bcbd5f51f)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:49.887930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a5f884-5f8c-4ccd-8b45-6ba2f4dfa55e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:50.085159 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-66a5f884-5f8c-4ccd-8b45-6ba2f4dfa55e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1df46d574e5641389fe6372bcbd5f51f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:50.087271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-66a5f884-5f8c-4ccd-8b45-6ba2f4dfa55e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1df46d574e5641389fe6372bcbd5f51f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:50.089036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-66a5f884-5f8c-4ccd-8b45-6ba2f4dfa55e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1df46d574e5641389fe6372bcbd5f51f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:50.118106 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-66a5f884-5f8c-4ccd-8b45-6ba2f4dfa55e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1df46d574e5641389fe6372bcbd5f51f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:50.144913 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-66a5f884-5f8c-4ccd-8b45-6ba2f4dfa55e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1df46d574e5641389fe6372bcbd5f51f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:50.250992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 866/1763] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:49 2022] DELETE /identity/v3/users/1df46d574e5641389fe6372bcbd5f51f => generated 0 bytes in 877 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:50.256901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fd41c-1034-44ee-8ab3-5319ca8e3800 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:50.257786 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fd41c-1034-44ee-8ab3-5319ca8e3800 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:50.258241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fd41c-1034-44ee-8ab3-5319ca8e3800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:50.360403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f23fd41c-1034-44ee-8ab3-5319ca8e3800 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:50.369983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f23fd41c-1034-44ee-8ab3-5319ca8e3800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:50.836578 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 898/1764] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 581 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:50.974259 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5f884-5f8c-4ccd-8b45-6ba2f4dfa55e admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:50.976068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-66a5f884-5f8c-4ccd-8b45-6ba2f4dfa55e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:51.133517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:51.324737 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5f884-5f8c-4ccd-8b45-6ba2f4dfa55e admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:51.342494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec6fc8-38e7-4c95-8115-66fdaa169f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:51.343425 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec6fc8-38e7-4c95-8115-66fdaa169f3b service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:51.345002 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec6fc8-38e7-4c95-8115-66fdaa169f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:51.347297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec6fc8-38e7-4c95-8115-66fdaa169f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:51.386565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec6fc8-38e7-4c95-8115-66fdaa169f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:51.391311 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec6fc8-38e7-4c95-8115-66fdaa169f3b service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:51.420252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d7e14-b1c0-4027-ba36-1bf1f149c132 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:51.422100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d7e14-b1c0-4027-ba36-1bf1f149c132 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:51.422577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d7e14-b1c0-4027-ba36-1bf1f149c132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:51.452571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 867/1765] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:51.455667 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c52d7e14-b1c0-4027-ba36-1bf1f149c132 None None] MFA Rules not processed for user `9aa753d1e2c4430498d5b28f0304952f`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:51.456662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78af55-7f52-4254-9833-3190bce60e8b None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:51.459554 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c52d7e14-b1c0-4027-ba36-1bf1f149c132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:51.461187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78af55-7f52-4254-9833-3190bce60e8b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:51.465603 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78af55-7f52-4254-9833-3190bce60e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:51.543897 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-7e78af55-7f52-4254-9833-3190bce60e8b None None] MFA Rules not processed for user `34fcb5885325472bbe338cf05d3788f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:51.548268 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7e78af55-7f52-4254-9833-3190bce60e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:51.583812 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 868/1766] 10.222.0.57 () {58 vars in 1015 bytes} [Wed Aug 24 12:32:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:51.634425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 899/1767] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:51.664476 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78af55-7f52-4254-9833-3190bce60e8b None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:51.667182 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7e78af55-7f52-4254-9833-3190bce60e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:51.688475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78af55-7f52-4254-9833-3190bce60e8b None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:51.700385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd48c7-8000-471d-9c41-92b6fb99702e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:51.703991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd48c7-8000-471d-9c41-92b6fb99702e service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:51.705803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd48c7-8000-471d-9c41-92b6fb99702e service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:51.707530 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd48c7-8000-471d-9c41-92b6fb99702e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:51.722894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd48c7-8000-471d-9c41-92b6fb99702e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:51.726423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fd48c7-8000-471d-9c41-92b6fb99702e service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:51.743653 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 869/1768] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:52.360235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d7e14-b1c0-4027-ba36-1bf1f149c132 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:52.364976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c52d7e14-b1c0-4027-ba36-1bf1f149c132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:52.429934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d7e14-b1c0-4027-ba36-1bf1f149c132 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:52.442180 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd48c7-8000-471d-9c41-92b6fb99702e service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:52.444365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-41fd48c7-8000-471d-9c41-92b6fb99702e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:52.452085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18717b8d-bffe-4eb5-9300-d7f3407b3f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:52.454499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18717b8d-bffe-4eb5-9300-d7f3407b3f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:52.456278 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18717b8d-bffe-4eb5-9300-d7f3407b3f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:52.456711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18717b8d-bffe-4eb5-9300-d7f3407b3f80 admin admin] PATH_INFO: `/v3/projects/98bc9d2fce244b0890a847f3ef9e4169` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:52.459370 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18717b8d-bffe-4eb5-9300-d7f3407b3f80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98bc9d2fce244b0890a847f3ef9e4169)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:52.463495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18717b8d-bffe-4eb5-9300-d7f3407b3f80 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:52.479121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd48c7-8000-471d-9c41-92b6fb99702e service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:52.551612 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c990a-6bc7-43bb-857f-d463eeffa315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:52.554859 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c990a-6bc7-43bb-857f-d463eeffa315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:52.561303 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c990a-6bc7-43bb-857f-d463eeffa315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:52.562592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c990a-6bc7-43bb-857f-d463eeffa315 admin admin] PATH_INFO: `/v3/users/d4026e21eeaf46888d239a01d8275aa9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:52.567975 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c990a-6bc7-43bb-857f-d463eeffa315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4026e21eeaf46888d239a01d8275aa9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:52.571987 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c990a-6bc7-43bb-857f-d463eeffa315 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:52.601925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-18717b8d-bffe-4eb5-9300-d7f3407b3f80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98bc9d2fce244b0890a847f3ef9e4169', 'request_id': 'req-18717b8d-bffe-4eb5-9300-d7f3407b3f80'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.649725 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-18717b8d-bffe-4eb5-9300-d7f3407b3f80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98bc9d2fce244b0890a847f3ef9e4169', 'request_id': 'req-18717b8d-bffe-4eb5-9300-d7f3407b3f80'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.688947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-18717b8d-bffe-4eb5-9300-d7f3407b3f80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98bc9d2fce244b0890a847f3ef9e4169', 'request_id': 'req-18717b8d-bffe-4eb5-9300-d7f3407b3f80'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.743500 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-18717b8d-bffe-4eb5-9300-d7f3407b3f80 admin admin] The token cache is being invalidate because project 98bc9d2fce244b0890a847f3ef9e4169 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:52.746662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-18717b8d-bffe-4eb5-9300-d7f3407b3f80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.748867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-18717b8d-bffe-4eb5-9300-d7f3407b3f80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.760829 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 900/1769] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:52 2022] DELETE /identity/v3/projects/98bc9d2fce244b0890a847f3ef9e4169 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:52.763674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711b4be-3dd0-49fd-8eb2-081f736349bc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:52.767770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711b4be-3dd0-49fd-8eb2-081f736349bc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:52.769228 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711b4be-3dd0-49fd-8eb2-081f736349bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:52.804676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-662c990a-6bc7-43bb-857f-d463eeffa315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4026e21eeaf46888d239a01d8275aa9'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.806559 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-662c990a-6bc7-43bb-857f-d463eeffa315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4026e21eeaf46888d239a01d8275aa9'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.808085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-662c990a-6bc7-43bb-857f-d463eeffa315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4026e21eeaf46888d239a01d8275aa9'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.821721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-662c990a-6bc7-43bb-857f-d463eeffa315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4026e21eeaf46888d239a01d8275aa9'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.833716 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-662c990a-6bc7-43bb-857f-d463eeffa315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4026e21eeaf46888d239a01d8275aa9'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:52.841467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1711b4be-3dd0-49fd-8eb2-081f736349bc None None] MFA Rules not processed for user `9138ac21bbdb4577a7f43b5ab471d572`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:52.845500 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1711b4be-3dd0-49fd-8eb2-081f736349bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:52.867821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 870/1770] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:52 2022] DELETE /identity/v3/users/d4026e21eeaf46888d239a01d8275aa9 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:52.869117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c990a-6bc7-43bb-857f-d463eeffa315 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:52.871242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-662c990a-6bc7-43bb-857f-d463eeffa315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:52.887683 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:52.985651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 901/1771] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:52 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:52.987351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7cc9d-abce-42c6-b4bb-613232473736 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:52.987797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7cc9d-abce-42c6-b4bb-613232473736 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:52.988186 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7cc9d-abce-42c6-b4bb-613232473736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:53.025549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c990a-6bc7-43bb-857f-d463eeffa315 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:53.034112 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-aec7cc9d-abce-42c6-b4bb-613232473736 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:53.035852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-aec7cc9d-abce-42c6-b4bb-613232473736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:53.053707 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81efc3-51c1-4e17-9750-a4f187cc62bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:53.058091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81efc3-51c1-4e17-9750-a4f187cc62bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:53.058518 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81efc3-51c1-4e17-9750-a4f187cc62bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:53.058831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81efc3-51c1-4e17-9750-a4f187cc62bc admin admin] PATH_INFO: `/v3/users/ee32af7b9a1243e3b8258a67d161c8bc` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:53.065466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81efc3-51c1-4e17-9750-a4f187cc62bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee32af7b9a1243e3b8258a67d161c8bc)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:53.076648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf81efc3-51c1-4e17-9750-a4f187cc62bc admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:53.080026 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 902/1772] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:53.094262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7cc9d-abce-42c6-b4bb-613232473736 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:53.097034 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-aec7cc9d-abce-42c6-b4bb-613232473736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:53.136217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:53.258624 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-bf81efc3-51c1-4e17-9750-a4f187cc62bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee32af7b9a1243e3b8258a67d161c8bc'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:53.281266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-bf81efc3-51c1-4e17-9750-a4f187cc62bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee32af7b9a1243e3b8258a67d161c8bc'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:53.282456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-bf81efc3-51c1-4e17-9750-a4f187cc62bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee32af7b9a1243e3b8258a67d161c8bc'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:53.297587 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-bf81efc3-51c1-4e17-9750-a4f187cc62bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee32af7b9a1243e3b8258a67d161c8bc'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:53.329619 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-bf81efc3-51c1-4e17-9750-a4f187cc62bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee32af7b9a1243e3b8258a67d161c8bc'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:53.350117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7cc9d-abce-42c6-b4bb-613232473736 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:53.363700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfecadc5-d53d-4b32-a62c-a66677c6b0c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:53.365723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecadc5-d53d-4b32-a62c-a66677c6b0c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:53.366042 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecadc5-d53d-4b32-a62c-a66677c6b0c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:53.366338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecadc5-d53d-4b32-a62c-a66677c6b0c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:53.389115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-dfecadc5-d53d-4b32-a62c-a66677c6b0c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:53.390926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 871/1773] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:32:52 2022] DELETE /identity/v3/users/ee32af7b9a1243e3b8258a67d161c8bc => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:53.432783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfecadc5-d53d-4b32-a62c-a66677c6b0c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:53.439114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfecadc5-d53d-4b32-a62c-a66677c6b0c8 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:53.470972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 903/1774] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:32:54.193497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81efc3-51c1-4e17-9750-a4f187cc62bc admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:54.196067 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-bf81efc3-51c1-4e17-9750-a4f187cc62bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:54.397801 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf81efc3-51c1-4e17-9750-a4f187cc62bc admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:54.410784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a41a3-04a0-423f-97a6-e93a2c15d389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:54.421174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a41a3-04a0-423f-97a6-e93a2c15d389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:54.421174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a41a3-04a0-423f-97a6-e93a2c15d389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:54.421174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a41a3-04a0-423f-97a6-e93a2c15d389 admin admin] PATH_INFO: `/v3/projects/c0a1fcc10cf44e0eb74b20e0afb9a0d0` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:54.421174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a41a3-04a0-423f-97a6-e93a2c15d389 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0a1fcc10cf44e0eb74b20e0afb9a0d0)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:54.424614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a41a3-04a0-423f-97a6-e93a2c15d389 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:54.680679 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d66a41a3-04a0-423f-97a6-e93a2c15d389 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0a1fcc10cf44e0eb74b20e0afb9a0d0', 'request_id': 'req-d66a41a3-04a0-423f-97a6-e93a2c15d389'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.695580 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d66a41a3-04a0-423f-97a6-e93a2c15d389 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0a1fcc10cf44e0eb74b20e0afb9a0d0', 'request_id': 'req-d66a41a3-04a0-423f-97a6-e93a2c15d389'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.735586 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d66a41a3-04a0-423f-97a6-e93a2c15d389 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0a1fcc10cf44e0eb74b20e0afb9a0d0', 'request_id': 'req-d66a41a3-04a0-423f-97a6-e93a2c15d389'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.769055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d66a41a3-04a0-423f-97a6-e93a2c15d389 admin admin] The token cache is being invalidate because project c0a1fcc10cf44e0eb74b20e0afb9a0d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:54.769420 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d66a41a3-04a0-423f-97a6-e93a2c15d389 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.778872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d66a41a3-04a0-423f-97a6-e93a2c15d389 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:54.788600 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76837de5-25f4-435c-baa6-f548f57b6763 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:54.789184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76837de5-25f4-435c-baa6-f548f57b6763 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:54.789184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76837de5-25f4-435c-baa6-f548f57b6763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:54.791854 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 872/1775] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:54 2022] DELETE /identity/v3/projects/c0a1fcc10cf44e0eb74b20e0afb9a0d0 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:54.818506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-76837de5-25f4-435c-baa6-f548f57b6763 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:54.823815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-76837de5-25f4-435c-baa6-f548f57b6763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:55.027655 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 904/1776] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:55.028900 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7b3f7-e22e-4be4-a4cd-cba717c2b058 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.029332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7b3f7-e22e-4be4-a4cd-cba717c2b058 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.029613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7b3f7-e22e-4be4-a4cd-cba717c2b058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.089793 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-7ab7b3f7-e22e-4be4-a4cd-cba717c2b058 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:55.095744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7ab7b3f7-e22e-4be4-a4cd-cba717c2b058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:55.140557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76837de5-25f4-435c-baa6-f548f57b6763 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.153203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 873/1777] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:55.155375 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76837de5-25f4-435c-baa6-f548f57b6763 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.192818 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6098cf-4a7b-4d00-a016-f0207bebb1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.193525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6098cf-4a7b-4d00-a016-f0207bebb1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.193792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6098cf-4a7b-4d00-a016-f0207bebb1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.196050 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6098cf-4a7b-4d00-a016-f0207bebb1d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.196763 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6098cf-4a7b-4d00-a016-f0207bebb1d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:55.197076 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6098cf-4a7b-4d00-a016-f0207bebb1d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.198832 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6098cf-4a7b-4d00-a016-f0207bebb1d5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.204904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7b3f7-e22e-4be4-a4cd-cba717c2b058 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.206651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7ab7b3f7-e22e-4be4-a4cd-cba717c2b058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:55.241464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7b3f7-e22e-4be4-a4cd-cba717c2b058 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.269661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e73266-d32e-4e52-8bd9-9bc574e910dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.270324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e73266-d32e-4e52-8bd9-9bc574e910dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.276956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 905/1778] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:55.278912 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e73266-d32e-4e52-8bd9-9bc574e910dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.280771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e73266-d32e-4e52-8bd9-9bc574e910dd admin admin] PATH_INFO: `/v3/projects/b777488b8bf94b5fb8ac61300cea8b37` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.283781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e73266-d32e-4e52-8bd9-9bc574e910dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b777488b8bf94b5fb8ac61300cea8b37)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.285645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6098cf-4a7b-4d00-a016-f0207bebb1d5 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.288656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e73266-d32e-4e52-8bd9-9bc574e910dd admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.329838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:55.339526 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6098cf-4a7b-4d00-a016-f0207bebb1d5 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.359083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b8fe1-69ab-4716-96d9-f2273444d777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.359952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b8fe1-69ab-4716-96d9-f2273444d777 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.361468 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b8fe1-69ab-4716-96d9-f2273444d777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.361831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b8fe1-69ab-4716-96d9-f2273444d777 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.362588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b8fe1-69ab-4716-96d9-f2273444d777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.370092 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b8fe1-69ab-4716-96d9-f2273444d777 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.466259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 906/1779] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:55 2022] POST /identity/v3/projects => generated 369 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:55.467987 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b8fe1-69ab-4716-96d9-f2273444d777 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.493148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-64e73266-d32e-4e52-8bd9-9bc574e910dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b777488b8bf94b5fb8ac61300cea8b37', 'request_id': 'req-64e73266-d32e-4e52-8bd9-9bc574e910dd'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:55.512937 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-64e73266-d32e-4e52-8bd9-9bc574e910dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b777488b8bf94b5fb8ac61300cea8b37', 'request_id': 'req-64e73266-d32e-4e52-8bd9-9bc574e910dd'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:55.539427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-64e73266-d32e-4e52-8bd9-9bc574e910dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b777488b8bf94b5fb8ac61300cea8b37', 'request_id': 'req-64e73266-d32e-4e52-8bd9-9bc574e910dd'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:55.558463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:55.571919 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-64e73266-d32e-4e52-8bd9-9bc574e910dd admin admin] The token cache is being invalidate because project b777488b8bf94b5fb8ac61300cea8b37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:32:55.581181 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-64e73266-d32e-4e52-8bd9-9bc574e910dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:55.582083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-64e73266-d32e-4e52-8bd9-9bc574e910dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:32:55.588716 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 874/1780] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:32:55 2022] DELETE /identity/v3/projects/b777488b8bf94b5fb8ac61300cea8b37 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:55.590352 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077e2a3-c2b1-412b-9313-8868e6e57868 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.590648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077e2a3-c2b1-412b-9313-8868e6e57868 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.590936 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077e2a3-c2b1-412b-9313-8868e6e57868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.632728 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b8fe1-69ab-4716-96d9-f2273444d777 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.639782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838e95d-efc9-454c-8783-7cf576ce4bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.640818 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838e95d-efc9-454c-8783-7cf576ce4bfb admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.641246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838e95d-efc9-454c-8783-7cf576ce4bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.641791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838e95d-efc9-454c-8783-7cf576ce4bfb admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.642617 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838e95d-efc9-454c-8783-7cf576ce4bfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.645509 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b838e95d-efc9-454c-8783-7cf576ce4bfb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.658675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-9077e2a3-c2b1-412b-9313-8868e6e57868 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:55.666979 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-9077e2a3-c2b1-412b-9313-8868e6e57868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:55.703683 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 875/1781] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:55.706669 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76720e17-4e38-489f-a7e5-11d2e930c63f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.706971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76720e17-4e38-489f-a7e5-11d2e930c63f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.707427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76720e17-4e38-489f-a7e5-11d2e930c63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.737461 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 907/1782] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:55 2022] POST /identity/v3/users => generated 285 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:55.743064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838e95d-efc9-454c-8783-7cf576ce4bfb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.758370 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b838e95d-efc9-454c-8783-7cf576ce4bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:55.775404 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838e95d-efc9-454c-8783-7cf576ce4bfb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.783042 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a759fb2-79cc-40af-a8fe-eb99e9c13334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.783860 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a759fb2-79cc-40af-a8fe-eb99e9c13334 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.784178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a759fb2-79cc-40af-a8fe-eb99e9c13334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.784495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a759fb2-79cc-40af-a8fe-eb99e9c13334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.785156 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a759fb2-79cc-40af-a8fe-eb99e9c13334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.787500 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a759fb2-79cc-40af-a8fe-eb99e9c13334 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.800320 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-76720e17-4e38-489f-a7e5-11d2e930c63f None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:55.801573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-76720e17-4e38-489f-a7e5-11d2e930c63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:55.803022 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 908/1783] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:32:55.808450 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a759fb2-79cc-40af-a8fe-eb99e9c13334 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.823675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 876/1784] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:55.834670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76720e17-4e38-489f-a7e5-11d2e930c63f None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.853717 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a759fb2-79cc-40af-a8fe-eb99e9c13334 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.875662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76720e17-4e38-489f-a7e5-11d2e930c63f None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:55.883143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e74d52-346b-4864-8742-af97b44b4daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.883855 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e74d52-346b-4864-8742-af97b44b4daf admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.884161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e74d52-346b-4864-8742-af97b44b4daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.884420 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e74d52-346b-4864-8742-af97b44b4daf admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.885083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e74d52-346b-4864-8742-af97b44b4daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.889327 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e74d52-346b-4864-8742-af97b44b4daf admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.898543 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c69c30-1828-4fdb-a3e8-9b57b92350d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:55.900445 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c69c30-1828-4fdb-a3e8-9b57b92350d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:55.901953 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c69c30-1828-4fdb-a3e8-9b57b92350d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:55.902980 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c69c30-1828-4fdb-a3e8-9b57b92350d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:55.911350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c69c30-1828-4fdb-a3e8-9b57b92350d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:32:55.911697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c69c30-1828-4fdb-a3e8-9b57b92350d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:55.916393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c69c30-1828-4fdb-a3e8-9b57b92350d5 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:55.955551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 909/1785] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:55.960425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e74d52-346b-4864-8742-af97b44b4daf admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.976553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 877/1786] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:32:55.981373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c69c30-1828-4fdb-a3e8-9b57b92350d5 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:55.991487 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e74d52-346b-4864-8742-af97b44b4daf admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:56.002899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c69c30-1828-4fdb-a3e8-9b57b92350d5 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:56.011431 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98458325-350b-49fb-b059-80de9ea728f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:56.014408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98458325-350b-49fb-b059-80de9ea728f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:56.014867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98458325-350b-49fb-b059-80de9ea728f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:56.015463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98458325-350b-49fb-b059-80de9ea728f4 admin admin] PATH_INFO: `/v3/projects/bd145b13ed6944e88bba901b715e79dd/users/d70aba4656f84f088f8609e985323e5e/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:56.023458 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638695d6-eb75-4800-8636-699bf1103da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:56.026801 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638695d6-eb75-4800-8636-699bf1103da9 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:56.027094 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638695d6-eb75-4800-8636-699bf1103da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:56.027369 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638695d6-eb75-4800-8636-699bf1103da9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:56.028043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638695d6-eb75-4800-8636-699bf1103da9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:56.041258 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638695d6-eb75-4800-8636-699bf1103da9 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:56.060643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98458325-350b-49fb-b059-80de9ea728f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd145b13ed6944e88bba901b715e79dd, user_id=d70aba4656f84f088f8609e985323e5e, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:56.065778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98458325-350b-49fb-b059-80de9ea728f4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:56.136453 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 910/1787] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:55 2022] PUT /identity/v3/projects/bd145b13ed6944e88bba901b715e79dd/users/d70aba4656f84f088f8609e985323e5e/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:56.140914 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee67e1b-3a5d-4daa-900b-9af4215d60df None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:56.142127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee67e1b-3a5d-4daa-900b-9af4215d60df None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:56.143166 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee67e1b-3a5d-4daa-900b-9af4215d60df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:56.182822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 878/1788] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:55 2022] POST /identity/v3/projects => generated 383 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:56.200296 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638695d6-eb75-4800-8636-699bf1103da9 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:56.300987 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1ee67e1b-3a5d-4daa-900b-9af4215d60df None None] MFA Rules not processed for user `d70aba4656f84f088f8609e985323e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:56.306485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1ee67e1b-3a5d-4daa-900b-9af4215d60df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:56.622117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 911/1789] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:56.623894 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee67e1b-3a5d-4daa-900b-9af4215d60df None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:56.624258 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638695d6-eb75-4800-8636-699bf1103da9 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:56.646233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee67e1b-3a5d-4daa-900b-9af4215d60df None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:56.652730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc8430-bd14-4f4a-b52d-6dc9ab5a92e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:56.655333 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc8430-bd14-4f4a-b52d-6dc9ab5a92e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:56.655675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc8430-bd14-4f4a-b52d-6dc9ab5a92e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:56.655983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc8430-bd14-4f4a-b52d-6dc9ab5a92e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:56.656693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc8430-bd14-4f4a-b52d-6dc9ab5a92e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:56.659505 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc8430-bd14-4f4a-b52d-6dc9ab5a92e4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:56.662728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57b86b-cc35-4024-a6fd-992107e7e6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:56.671492 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57b86b-cc35-4024-a6fd-992107e7e6a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:56.676370 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57b86b-cc35-4024-a6fd-992107e7e6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:56.679233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57b86b-cc35-4024-a6fd-992107e7e6a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:56.682666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57b86b-cc35-4024-a6fd-992107e7e6a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:56.693509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57b86b-cc35-4024-a6fd-992107e7e6a0 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:56.776213 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 912/1790] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:56 2022] POST /identity/v3/projects => generated 367 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:56.778186 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9e4eb-f0f1-40af-ab2b-2be7a4dc8f64 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:56.778631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9e4eb-f0f1-40af-ab2b-2be7a4dc8f64 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:56.780283 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9e4eb-f0f1-40af-ab2b-2be7a4dc8f64 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:56.785309 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 913/1791] 10.222.0.58 () {54 vars in 921 bytes} [Wed Aug 24 12:32:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:32:56.787408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9e4eb-f0f1-40af-ab2b-2be7a4dc8f64 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:56.828845 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 879/1792] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:56 2022] POST /identity/v3/users => generated 292 bytes in 628 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:56.830203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57b86b-cc35-4024-a6fd-992107e7e6a0 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:56.892333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57b86b-cc35-4024-a6fd-992107e7e6a0 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:56.987444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:56.990718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:57.053852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9e4eb-f0f1-40af-ab2b-2be7a4dc8f64 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:57.065080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451ca8c-79a9-4814-829d-11e51ead8397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:57.065737 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451ca8c-79a9-4814-829d-11e51ead8397 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:57.066022 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451ca8c-79a9-4814-829d-11e51ead8397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:57.066301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451ca8c-79a9-4814-829d-11e51ead8397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:57.067025 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451ca8c-79a9-4814-829d-11e51ead8397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:57.068652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f6de2-5056-49b0-aacf-d4847baf6e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:57.069278 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f6de2-5056-49b0-aacf-d4847baf6e74 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:57.069559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f6de2-5056-49b0-aacf-d4847baf6e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:57.069892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f6de2-5056-49b0-aacf-d4847baf6e74 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:57.070542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f6de2-5056-49b0-aacf-d4847baf6e74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:57.073309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451ca8c-79a9-4814-829d-11e51ead8397 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:57.075023 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f6de2-5056-49b0-aacf-d4847baf6e74 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:57.096294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 880/1793] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:32:57.096294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b53013-a21e-4314-8519-32c2132093a2 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:57.096294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b53013-a21e-4314-8519-32c2132093a2 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:57.096294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b53013-a21e-4314-8519-32c2132093a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:57.147036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-b8b53013-a21e-4314-8519-32c2132093a2 None None] MFA Rules not processed for user `311dbed3ab0b44a7830a93227ff9ad33`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:57.148447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b8b53013-a21e-4314-8519-32c2132093a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:57.150481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 914/1794] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:56 2022] POST /identity/v3/users => generated 290 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:57.151416 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f6de2-5056-49b0-aacf-d4847baf6e74 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:57.191755 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f6de2-5056-49b0-aacf-d4847baf6e74 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:57.204972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde413fa-f5f3-4366-a821-aacaaac44d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:57.206118 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde413fa-f5f3-4366-a821-aacaaac44d70 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:57.210453 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde413fa-f5f3-4366-a821-aacaaac44d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:57.210926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde413fa-f5f3-4366-a821-aacaaac44d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:57.211646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde413fa-f5f3-4366-a821-aacaaac44d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:57.219882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde413fa-f5f3-4366-a821-aacaaac44d70 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:57.276181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 915/1795] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:57.286504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde413fa-f5f3-4366-a821-aacaaac44d70 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:57.319695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 881/1796] 10.222.0.58 () {58 vars in 1018 bytes} [Wed Aug 24 12:32:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:57.320858 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b53013-a21e-4314-8519-32c2132093a2 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:57.323819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde413fa-f5f3-4366-a821-aacaaac44d70 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:57.336197 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b53013-a21e-4314-8519-32c2132093a2 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:57.345626 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bea44e-2947-4bb2-9534-39d45e1ce68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:57.346674 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bea44e-2947-4bb2-9534-39d45e1ce68b admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:57.349173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bea44e-2947-4bb2-9534-39d45e1ce68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:57.349463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bea44e-2947-4bb2-9534-39d45e1ce68b admin admin] PATH_INFO: `/v3/projects/156ce21af05e4b378a5851367cb7d508/users/2c0edaaff9f543e6955df2e07e86054d/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:57.356376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae586a7e-a5a5-4bef-954f-009a8012adfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:57.358382 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae586a7e-a5a5-4bef-954f-009a8012adfb admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:57.360157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae586a7e-a5a5-4bef-954f-009a8012adfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:57.360481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae586a7e-a5a5-4bef-954f-009a8012adfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:57.364960 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae586a7e-a5a5-4bef-954f-009a8012adfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:57.368272 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae586a7e-a5a5-4bef-954f-009a8012adfb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:57.393759 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bea44e-2947-4bb2-9534-39d45e1ce68b admin admin] RBAC: Authorizing `identity:create_grant(project_id=156ce21af05e4b378a5851367cb7d508, user_id=2c0edaaff9f543e6955df2e07e86054d, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:57.400726 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 916/1797] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:57.402884 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae586a7e-a5a5-4bef-954f-009a8012adfb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:57.405450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bea44e-2947-4bb2-9534-39d45e1ce68b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:57.428496 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae586a7e-a5a5-4bef-954f-009a8012adfb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:57.434739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 882/1798] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:57 2022] PUT /identity/v3/projects/156ce21af05e4b378a5851367cb7d508/users/2c0edaaff9f543e6955df2e07e86054d/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:57.447417 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f0560-f785-4838-8783-50b452d557c3 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:57.448052 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f0560-f785-4838-8783-50b452d557c3 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:57.448356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f0560-f785-4838-8783-50b452d557c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:57.522637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-349f0560-f785-4838-8783-50b452d557c3 None None] MFA Rules not processed for user `2c0edaaff9f543e6955df2e07e86054d`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:57.528298 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-349f0560-f785-4838-8783-50b452d557c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:57.590579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a816c46-95ae-4a78-a647-eaafb06decfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:57.591372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a816c46-95ae-4a78-a647-eaafb06decfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:57.592385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a816c46-95ae-4a78-a647-eaafb06decfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:57.592645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a816c46-95ae-4a78-a647-eaafb06decfc admin admin] PATH_INFO: `/v3/projects/9e0afb7efd194e7bb9ef17f793dc7d55/users/8ef60a46cc0545479f4ea7abda6bcfe5/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:57.630892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a816c46-95ae-4a78-a647-eaafb06decfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0afb7efd194e7bb9ef17f793dc7d55, user_id=8ef60a46cc0545479f4ea7abda6bcfe5, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:57.650675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a816c46-95ae-4a78-a647-eaafb06decfc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:57.686268 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 917/1799] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:57 2022] PUT /identity/v3/projects/9e0afb7efd194e7bb9ef17f793dc7d55/users/8ef60a46cc0545479f4ea7abda6bcfe5/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:57.697904 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a816c46-95ae-4a78-a647-eaafb06decfc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:57.747122 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 883/1800] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:32:57.750338 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f0560-f785-4838-8783-50b452d557c3 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:57.790311 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f0560-f785-4838-8783-50b452d557c3 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:57.916641 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:57.954679 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a816c46-95ae-4a78-a647-eaafb06decfc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:57.978910 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a748d0-b6b4-454b-8a28-1b8254f3cb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:57.981018 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a748d0-b6b4-454b-8a28-1b8254f3cb93 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:57.981585 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a748d0-b6b4-454b-8a28-1b8254f3cb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:57.982105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a748d0-b6b4-454b-8a28-1b8254f3cb93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:57.985394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a748d0-b6b4-454b-8a28-1b8254f3cb93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:57.988263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a748d0-b6b4-454b-8a28-1b8254f3cb93 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:58.005222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 918/1801] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:58.010160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a748d0-b6b4-454b-8a28-1b8254f3cb93 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:58.011161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61f981-7a5f-42fc-b690-7ebe40bcec04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:58.011615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61f981-7a5f-42fc-b690-7ebe40bcec04 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.011991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61f981-7a5f-42fc-b690-7ebe40bcec04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.012196 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61f981-7a5f-42fc-b690-7ebe40bcec04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.012842 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61f981-7a5f-42fc-b690-7ebe40bcec04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:58.015474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61f981-7a5f-42fc-b690-7ebe40bcec04 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:58.028664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a748d0-b6b4-454b-8a28-1b8254f3cb93 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:58.047698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe6ac0-f2d0-4445-873f-b3652d66d16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:58.048485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe6ac0-f2d0-4445-873f-b3652d66d16b admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.048859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe6ac0-f2d0-4445-873f-b3652d66d16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.049142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe6ac0-f2d0-4445-873f-b3652d66d16b admin admin] PATH_INFO: `/v3/projects/9e0afb7efd194e7bb9ef17f793dc7d55/users/8ef60a46cc0545479f4ea7abda6bcfe5/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.060866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe6ac0-f2d0-4445-873f-b3652d66d16b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0afb7efd194e7bb9ef17f793dc7d55, user_id=8ef60a46cc0545479f4ea7abda6bcfe5, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:58.066802 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 884/1802] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:57 2022] POST /identity/v3/projects => generated 385 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:58.068021 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe6ac0-f2d0-4445-873f-b3652d66d16b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:58.071449 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61f981-7a5f-42fc-b690-7ebe40bcec04 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:58.155272 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 919/1803] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:58 2022] PUT /identity/v3/projects/9e0afb7efd194e7bb9ef17f793dc7d55/users/8ef60a46cc0545479f4ea7abda6bcfe5/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:32:58.156956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f404c-d41e-4aac-b97d-d6387457f38b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.157399 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f404c-d41e-4aac-b97d-d6387457f38b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.159496 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f404c-d41e-4aac-b97d-d6387457f38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.224967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-699f404c-d41e-4aac-b97d-d6387457f38b None None] MFA Rules not processed for user `8ef60a46cc0545479f4ea7abda6bcfe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:58.234758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-699f404c-d41e-4aac-b97d-d6387457f38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:58.249576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:58.358003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61f981-7a5f-42fc-b690-7ebe40bcec04 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:58.526932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 920/1804] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:58.535708 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f404c-d41e-4aac-b97d-d6387457f38b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:58.538493 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-699f404c-d41e-4aac-b97d-d6387457f38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:58.572315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1040b84-bf0f-4bfb-b631-3d9387851b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:58.574215 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1040b84-bf0f-4bfb-b631-3d9387851b2b admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.578259 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1040b84-bf0f-4bfb-b631-3d9387851b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.578732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1040b84-bf0f-4bfb-b631-3d9387851b2b admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.579468 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1040b84-bf0f-4bfb-b631-3d9387851b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:58.583730 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1040b84-bf0f-4bfb-b631-3d9387851b2b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:58.674643 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 885/1805] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:58 2022] POST /identity/v3/users => generated 293 bytes in 603 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:58.682968 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1040b84-bf0f-4bfb-b631-3d9387851b2b admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:58.718783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1040b84-bf0f-4bfb-b631-3d9387851b2b admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:58.731879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c16bc5-87d7-473a-b4c6-869704f886e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:58.733169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c16bc5-87d7-473a-b4c6-869704f886e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.733445 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c16bc5-87d7-473a-b4c6-869704f886e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.733879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c16bc5-87d7-473a-b4c6-869704f886e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.734557 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c16bc5-87d7-473a-b4c6-869704f886e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:58.744105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c16bc5-87d7-473a-b4c6-869704f886e8 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:58.761444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f404c-d41e-4aac-b97d-d6387457f38b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:58.780127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975acb3-2c3c-4969-a201-601fd43b2641 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:58.780859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975acb3-2c3c-4969-a201-601fd43b2641 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.781142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975acb3-2c3c-4969-a201-601fd43b2641 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.781418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975acb3-2c3c-4969-a201-601fd43b2641 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.797353 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 886/1806] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:58.798651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c16bc5-87d7-473a-b4c6-869704f886e8 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:58.839672 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975acb3-2c3c-4969-a201-601fd43b2641 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:58.841157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975acb3-2c3c-4969-a201-601fd43b2641 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:58.848924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c16bc5-87d7-473a-b4c6-869704f886e8 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:58.871694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895f9ea-ff12-4621-954a-58b3deefc9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:58.872364 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895f9ea-ff12-4621-954a-58b3deefc9b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:58.872702 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895f9ea-ff12-4621-954a-58b3deefc9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:58.873035 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895f9ea-ff12-4621-954a-58b3deefc9b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:58.900155 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895f9ea-ff12-4621-954a-58b3deefc9b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:58.910862 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895f9ea-ff12-4621-954a-58b3deefc9b0 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:58.913641 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 921/1807] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:32:58.965346 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 887/1808] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:32:58.970854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975acb3-2c3c-4969-a201-601fd43b2641 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:58.990026 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975acb3-2c3c-4969-a201-601fd43b2641 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:59.010929 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0673e7-4c53-498c-84b7-eb95af8b5808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:59.015842 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0673e7-4c53-498c-84b7-eb95af8b5808 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.016291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0673e7-4c53-498c-84b7-eb95af8b5808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.016621 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0673e7-4c53-498c-84b7-eb95af8b5808 admin admin] PATH_INFO: `/v3/projects/1d6f575defed448684d597f86a4c25df/users/adb994037a454bc5b9ef5931077104e4/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.089071 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0673e7-4c53-498c-84b7-eb95af8b5808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d6f575defed448684d597f86a4c25df, user_id=adb994037a454bc5b9ef5931077104e4, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:59.095864 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0673e7-4c53-498c-84b7-eb95af8b5808 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:59.173235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 922/1809] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:32:58 2022] PUT /identity/v3/projects/1d6f575defed448684d597f86a4c25df/users/adb994037a454bc5b9ef5931077104e4/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:32:59.185029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715185b-5b98-4ac7-9f27-878b913ca91a None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.185789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715185b-5b98-4ac7-9f27-878b913ca91a None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.186454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715185b-5b98-4ac7-9f27-878b913ca91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.274088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-6715185b-5b98-4ac7-9f27-878b913ca91a None None] MFA Rules not processed for user `adb994037a454bc5b9ef5931077104e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:32:59.278570 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-6715185b-5b98-4ac7-9f27-878b913ca91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:59.333897 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:32:59.484713 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 888/1810] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:32:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:32:59.488998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0673e7-4c53-498c-84b7-eb95af8b5808 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:59.559646 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715185b-5b98-4ac7-9f27-878b913ca91a None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:59.562061 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-6715185b-5b98-4ac7-9f27-878b913ca91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:32:59.744638 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0673e7-4c53-498c-84b7-eb95af8b5808 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:59.756522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b053f5-de03-45ec-b656-e892198daea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:59.757150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b053f5-de03-45ec-b656-e892198daea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.757435 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b053f5-de03-45ec-b656-e892198daea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.757709 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b053f5-de03-45ec-b656-e892198daea2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.758701 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b053f5-de03-45ec-b656-e892198daea2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:32:59.760417 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b053f5-de03-45ec-b656-e892198daea2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:32:59.890939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715185b-5b98-4ac7-9f27-878b913ca91a None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:32:59.914656 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 923/1811] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:32:59 2022] POST /identity/v3/projects => generated 383 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:32:59.915909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b053f5-de03-45ec-b656-e892198daea2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:32:59.930753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06322f-2034-4d7a-814e-6d2ced6e855c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:32:59.931739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06322f-2034-4d7a-814e-6d2ced6e855c service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:32:59.934951 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06322f-2034-4d7a-814e-6d2ced6e855c service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:32:59.936071 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06322f-2034-4d7a-814e-6d2ced6e855c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:32:59.949128 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:00.008809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06322f-2034-4d7a-814e-6d2ced6e855c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:00.014571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f06322f-2034-4d7a-814e-6d2ced6e855c service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:00.086210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 889/1812] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:32:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:00.304389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b053f5-de03-45ec-b656-e892198daea2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:00.344015 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230c55f-f90f-43d5-863c-e89c05bd81a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:00.344749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230c55f-f90f-43d5-863c-e89c05bd81a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.345461 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230c55f-f90f-43d5-863c-e89c05bd81a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.346279 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230c55f-f90f-43d5-863c-e89c05bd81a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.348447 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230c55f-f90f-43d5-863c-e89c05bd81a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:00.365052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230c55f-f90f-43d5-863c-e89c05bd81a5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:00.450700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 924/1813] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:32:59 2022] POST /identity/v3/users => generated 298 bytes in 535 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:00.452715 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06322f-2034-4d7a-814e-6d2ced6e855c service nova] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:00.472148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06322f-2034-4d7a-814e-6d2ced6e855c service nova] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:00.493105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a503096-fe1a-4266-ae8d-fc55ce7ef043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:00.494119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a503096-fe1a-4266-ae8d-fc55ce7ef043 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.494568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a503096-fe1a-4266-ae8d-fc55ce7ef043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.494921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a503096-fe1a-4266-ae8d-fc55ce7ef043 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.495876 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a503096-fe1a-4266-ae8d-fc55ce7ef043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:00.504791 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a503096-fe1a-4266-ae8d-fc55ce7ef043 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:00.538445 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 890/1814] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:00.545603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230c55f-f90f-43d5-863c-e89c05bd81a5 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:00.575714 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230c55f-f90f-43d5-863c-e89c05bd81a5 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:00.588690 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7741f7-14ab-4b48-afbe-21ae94fadb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:00.592687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7741f7-14ab-4b48-afbe-21ae94fadb5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:00.593190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7741f7-14ab-4b48-afbe-21ae94fadb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:00.593477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7741f7-14ab-4b48-afbe-21ae94fadb5b admin admin] PATH_INFO: `/v3/projects/f513336a5d9141ce99f55c3b3bc08dd7/users/7db724de032448f79e478a022581724a/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:00.640990 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7741f7-14ab-4b48-afbe-21ae94fadb5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f513336a5d9141ce99f55c3b3bc08dd7, user_id=7db724de032448f79e478a022581724a, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:00.656859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7741f7-14ab-4b48-afbe-21ae94fadb5b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:00.722857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 925/1815] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:00 2022] PUT /identity/v3/projects/f513336a5d9141ce99f55c3b3bc08dd7/users/7db724de032448f79e478a022581724a/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:00.727792 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a503096-fe1a-4266-ae8d-fc55ce7ef043 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:00.755591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a503096-fe1a-4266-ae8d-fc55ce7ef043 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:01.059514 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa66a782-8ee3-4577-84f8-ea7289b07157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:01.063533 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66a782-8ee3-4577-84f8-ea7289b07157 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:01.063877 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66a782-8ee3-4577-84f8-ea7289b07157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:01.064164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66a782-8ee3-4577-84f8-ea7289b07157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:01.064852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66a782-8ee3-4577-84f8-ea7289b07157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:01.075201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa66a782-8ee3-4577-84f8-ea7289b07157 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:01.134624 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 891/1816] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:01.138882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7741f7-14ab-4b48-afbe-21ae94fadb5b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:01.184803 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7741f7-14ab-4b48-afbe-21ae94fadb5b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:01.213813 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07ab03-3cca-455f-8d2b-b210fff55654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:01.214371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07ab03-3cca-455f-8d2b-b210fff55654 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:01.215282 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07ab03-3cca-455f-8d2b-b210fff55654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:01.215766 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07ab03-3cca-455f-8d2b-b210fff55654 admin admin] PATH_INFO: `/v3/projects/f513336a5d9141ce99f55c3b3bc08dd7/users/7db724de032448f79e478a022581724a/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:01.224639 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07ab03-3cca-455f-8d2b-b210fff55654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f513336a5d9141ce99f55c3b3bc08dd7, user_id=7db724de032448f79e478a022581724a, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:01.234726 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07ab03-3cca-455f-8d2b-b210fff55654 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:01.274942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 926/1817] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:01 2022] PUT /identity/v3/projects/f513336a5d9141ce99f55c3b3bc08dd7/users/7db724de032448f79e478a022581724a/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:01.288640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792de97-d0f4-4ba1-81c6-24833cf0b53b None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:01.289178 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792de97-d0f4-4ba1-81c6-24833cf0b53b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:01.289603 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792de97-d0f4-4ba1-81c6-24833cf0b53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:01.402035 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-8792de97-d0f4-4ba1-81c6-24833cf0b53b None None] MFA Rules not processed for user `7db724de032448f79e478a022581724a`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:01.413455 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-8792de97-d0f4-4ba1-81c6-24833cf0b53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:01.522620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:01.807562 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 892/1818] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:01 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 520 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:01.951485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07ab03-3cca-455f-8d2b-b210fff55654 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:01.954490 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0a07ab03-3cca-455f-8d2b-b210fff55654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:02.173849 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07ab03-3cca-455f-8d2b-b210fff55654 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:02.192773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ac919-2c28-416c-bdb8-4a4ff907a508 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:02.194802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ac919-2c28-416c-bdb8-4a4ff907a508 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:02.194950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ac919-2c28-416c-bdb8-4a4ff907a508 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:02.196389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ac919-2c28-416c-bdb8-4a4ff907a508 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:02.378460 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ac919-2c28-416c-bdb8-4a4ff907a508 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:02.379871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ac919-2c28-416c-bdb8-4a4ff907a508 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:02.411270 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 927/1819] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:03.136536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792de97-d0f4-4ba1-81c6-24833cf0b53b None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:03.215945 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792de97-d0f4-4ba1-81c6-24833cf0b53b None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:03.222301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8c377-6847-4ec7-ba4a-ebf9d48db790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:03.223635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8c377-6847-4ec7-ba4a-ebf9d48db790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.223939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8c377-6847-4ec7-ba4a-ebf9d48db790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.224211 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8c377-6847-4ec7-ba4a-ebf9d48db790 admin admin] PATH_INFO: `/v3/users/d70aba4656f84f088f8609e985323e5e` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.225741 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8c377-6847-4ec7-ba4a-ebf9d48db790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d70aba4656f84f088f8609e985323e5e)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:03.227511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8c377-6847-4ec7-ba4a-ebf9d48db790 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:03.307337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7cd8c377-6847-4ec7-ba4a-ebf9d48db790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd70aba4656f84f088f8609e985323e5e'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:03.308963 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7cd8c377-6847-4ec7-ba4a-ebf9d48db790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd70aba4656f84f088f8609e985323e5e'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:03.309869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7cd8c377-6847-4ec7-ba4a-ebf9d48db790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd70aba4656f84f088f8609e985323e5e'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:03.323068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7cd8c377-6847-4ec7-ba4a-ebf9d48db790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd70aba4656f84f088f8609e985323e5e'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:03.338570 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7cd8c377-6847-4ec7-ba4a-ebf9d48db790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd70aba4656f84f088f8609e985323e5e'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:03.366241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 893/1820] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:03 2022] DELETE /identity/v3/users/d70aba4656f84f088f8609e985323e5e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:03.367951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb4847-0197-4c81-8c2a-1bf35e6be432 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.368536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb4847-0197-4c81-8c2a-1bf35e6be432 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.368536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb4847-0197-4c81-8c2a-1bf35e6be432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.393908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-abcb4847-0197-4c81-8c2a-1bf35e6be432 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:03.395416 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-abcb4847-0197-4c81-8c2a-1bf35e6be432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:03.401192 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:03.492487 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 928/1821] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:03.521869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8c377-6847-4ec7-ba4a-ebf9d48db790 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:03.524699 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7cd8c377-6847-4ec7-ba4a-ebf9d48db790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:03.588505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:03.615152 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8c377-6847-4ec7-ba4a-ebf9d48db790 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:03.619599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634c944-87ab-499d-a2a7-9a9630b6fdfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:03.621038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634c944-87ab-499d-a2a7-9a9630b6fdfe service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.621497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634c944-87ab-499d-a2a7-9a9630b6fdfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.621937 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634c944-87ab-499d-a2a7-9a9630b6fdfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.649865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634c944-87ab-499d-a2a7-9a9630b6fdfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:03.651359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634c944-87ab-499d-a2a7-9a9630b6fdfe service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:03.667835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 894/1822] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:03.869799 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d6b40-7bd4-4b7c-8333-101615caf13b None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.870567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d6b40-7bd4-4b7c-8333-101615caf13b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.871096 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d6b40-7bd4-4b7c-8333-101615caf13b None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.889784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 929/1823] 10.222.0.58 () {54 vars in 921 bytes} [Wed Aug 24 12:33:03 2022] GET /identity => generated 270 bytes in 21 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:33:03.903048 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f823d1-0080-4db8-bc90-96156235b79b None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:03.903784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f823d1-0080-4db8-bc90-96156235b79b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:03.904834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f823d1-0080-4db8-bc90-96156235b79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:03.938099 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-87f823d1-0080-4db8-bc90-96156235b79b None None] MFA Rules not processed for user `311dbed3ab0b44a7830a93227ff9ad33`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:03.939665 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-87f823d1-0080-4db8-bc90-96156235b79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:04.048591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 895/1824] 10.222.0.58 () {58 vars in 1018 bytes} [Wed Aug 24 12:33:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:04.391063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d6b40-7bd4-4b7c-8333-101615caf13b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:04.395109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-786d6b40-7bd4-4b7c-8333-101615caf13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:04.422872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d6b40-7bd4-4b7c-8333-101615caf13b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:04.429662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:04.430390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:04.430670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:04.431098 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785 admin admin] PATH_INFO: `/v3/projects/bd145b13ed6944e88bba901b715e79dd` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:04.432453 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd145b13ed6944e88bba901b715e79dd)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:04.434819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:04.447757 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f823d1-0080-4db8-bc90-96156235b79b None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:04.450541 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-87f823d1-0080-4db8-bc90-96156235b79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:04.468740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f823d1-0080-4db8-bc90-96156235b79b None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:04.474369 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064add16-b5c4-4d29-a82c-d933561efaa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:04.474982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064add16-b5c4-4d29-a82c-d933561efaa6 service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:04.475266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064add16-b5c4-4d29-a82c-d933561efaa6 service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:04.475536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064add16-b5c4-4d29-a82c-d933561efaa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:04.487047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064add16-b5c4-4d29-a82c-d933561efaa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:04.489569 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064add16-b5c4-4d29-a82c-d933561efaa6 service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:04.508205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 896/1825] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:04.515441 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd145b13ed6944e88bba901b715e79dd', 'request_id': 'req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:04.522802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd145b13ed6944e88bba901b715e79dd', 'request_id': 'req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:04.531905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd145b13ed6944e88bba901b715e79dd', 'request_id': 'req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:04.545120 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785 admin admin] The token cache is being invalidate because project bd145b13ed6944e88bba901b715e79dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:04.545586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:04.546278 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:04.547980 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 930/1826] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:04 2022] DELETE /identity/v3/projects/bd145b13ed6944e88bba901b715e79dd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:04.552158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064add16-b5c4-4d29-a82c-d933561efaa6 service nova] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:04.563240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064add16-b5c4-4d29-a82c-d933561efaa6 service nova] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:04.625413 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e61296-ea03-4d08-9627-e6e1513c6101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:04.626051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e61296-ea03-4d08-9627-e6e1513c6101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:04.626336 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e61296-ea03-4d08-9627-e6e1513c6101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:04.626615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e61296-ea03-4d08-9627-e6e1513c6101 admin admin] PATH_INFO: `/v3/users/8ef60a46cc0545479f4ea7abda6bcfe5` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:04.627775 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e61296-ea03-4d08-9627-e6e1513c6101 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ef60a46cc0545479f4ea7abda6bcfe5)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:04.629717 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e61296-ea03-4d08-9627-e6e1513c6101 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:04.684534 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-36e61296-ea03-4d08-9627-e6e1513c6101 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ef60a46cc0545479f4ea7abda6bcfe5'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:04.685953 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-36e61296-ea03-4d08-9627-e6e1513c6101 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ef60a46cc0545479f4ea7abda6bcfe5'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:04.686767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-36e61296-ea03-4d08-9627-e6e1513c6101 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ef60a46cc0545479f4ea7abda6bcfe5'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:04.694018 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-36e61296-ea03-4d08-9627-e6e1513c6101 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ef60a46cc0545479f4ea7abda6bcfe5'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:04.696533 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:04.703460 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-36e61296-ea03-4d08-9627-e6e1513c6101 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ef60a46cc0545479f4ea7abda6bcfe5'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:04.724609 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 897/1827] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:04 2022] DELETE /identity/v3/users/8ef60a46cc0545479f4ea7abda6bcfe5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:05.030875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:05.038886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:05.052875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5426f-1c05-4ca3-8797-5dd5ff0e2785 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:05.130933 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54855cb-3888-44eb-bda5-5f13f321fce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:05.132219 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54855cb-3888-44eb-bda5-5f13f321fce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:05.132368 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54855cb-3888-44eb-bda5-5f13f321fce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:05.132846 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54855cb-3888-44eb-bda5-5f13f321fce2 admin admin] PATH_INFO: `/v3/projects/9e0afb7efd194e7bb9ef17f793dc7d55` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:05.133812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54855cb-3888-44eb-bda5-5f13f321fce2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e0afb7efd194e7bb9ef17f793dc7d55)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:05.136238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54855cb-3888-44eb-bda5-5f13f321fce2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:05.182640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:05.195490 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e54855cb-3888-44eb-bda5-5f13f321fce2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e0afb7efd194e7bb9ef17f793dc7d55', 'request_id': 'req-e54855cb-3888-44eb-bda5-5f13f321fce2'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:05.207080 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e54855cb-3888-44eb-bda5-5f13f321fce2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e0afb7efd194e7bb9ef17f793dc7d55', 'request_id': 'req-e54855cb-3888-44eb-bda5-5f13f321fce2'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:05.221264 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e54855cb-3888-44eb-bda5-5f13f321fce2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e0afb7efd194e7bb9ef17f793dc7d55', 'request_id': 'req-e54855cb-3888-44eb-bda5-5f13f321fce2'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:05.276329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e54855cb-3888-44eb-bda5-5f13f321fce2 admin admin] The token cache is being invalidate because project 9e0afb7efd194e7bb9ef17f793dc7d55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:05.276329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e54855cb-3888-44eb-bda5-5f13f321fce2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:05.276329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e54855cb-3888-44eb-bda5-5f13f321fce2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:05.276329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 931/1828] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:05 2022] DELETE /identity/v3/projects/9e0afb7efd194e7bb9ef17f793dc7d55 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:05.279910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa518661-c57f-4419-88f9-f6f94ce7b46d None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:05.279910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa518661-c57f-4419-88f9-f6f94ce7b46d None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:05.279910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa518661-c57f-4419-88f9-f6f94ce7b46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:05.348601 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-fa518661-c57f-4419-88f9-f6f94ce7b46d None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:05.349864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fa518661-c57f-4419-88f9-f6f94ce7b46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:05.507702 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 898/1829] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:05.520573 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7ab1a-f982-4e12-a278-b93223e4c253 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:05.521682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7ab1a-f982-4e12-a278-b93223e4c253 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:05.522285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7ab1a-f982-4e12-a278-b93223e4c253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:05.557863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa518661-c57f-4419-88f9-f6f94ce7b46d None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:05.559212 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fa518661-c57f-4419-88f9-f6f94ce7b46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:05.579594 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a5a7ab1a-f982-4e12-a278-b93223e4c253 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:05.588539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a5a7ab1a-f982-4e12-a278-b93223e4c253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:05.616055 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 932/1830] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:05.624966 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7ab1a-f982-4e12-a278-b93223e4c253 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:05.652223 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7ab1a-f982-4e12-a278-b93223e4c253 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:05.674159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c09cf-3a25-4c1b-b517-f110bc5fb177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:05.675433 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c09cf-3a25-4c1b-b517-f110bc5fb177 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:05.675978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c09cf-3a25-4c1b-b517-f110bc5fb177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:05.676463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c09cf-3a25-4c1b-b517-f110bc5fb177 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:05.677363 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c09cf-3a25-4c1b-b517-f110bc5fb177 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:05.678335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c09cf-3a25-4c1b-b517-f110bc5fb177 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:05.680470 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c09cf-3a25-4c1b-b517-f110bc5fb177 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:05.713154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 933/1831] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:05.723829 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c09cf-3a25-4c1b-b517-f110bc5fb177 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:05.729235 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa518661-c57f-4419-88f9-f6f94ce7b46d None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:05.735041 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e19d8-a2ff-45e8-96fc-3567fa977d7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:05.736216 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e19d8-a2ff-45e8-96fc-3567fa977d7f service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:05.736216 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e19d8-a2ff-45e8-96fc-3567fa977d7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:05.736216 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e19d8-a2ff-45e8-96fc-3567fa977d7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:05.737543 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c09cf-3a25-4c1b-b517-f110bc5fb177 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:05.738345 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-686e19d8-a2ff-45e8-96fc-3567fa977d7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:05.742912 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d43645-c522-4f3a-b705-49de7c39d80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:05.743760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d43645-c522-4f3a-b705-49de7c39d80a admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:05.744218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d43645-c522-4f3a-b705-49de7c39d80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:05.744644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d43645-c522-4f3a-b705-49de7c39d80a admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:05.745434 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d43645-c522-4f3a-b705-49de7c39d80a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:05.748610 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d43645-c522-4f3a-b705-49de7c39d80a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:05.834931 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 934/1832] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:05 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:05.838888 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d43645-c522-4f3a-b705-49de7c39d80a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:05.902789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e19d8-a2ff-45e8-96fc-3567fa977d7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:05.905146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e19d8-a2ff-45e8-96fc-3567fa977d7f service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:05.941396 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 899/1833] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:05.996442 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d43645-c522-4f3a-b705-49de7c39d80a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:06.002182 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66c51f-0656-49ac-8297-19e448835674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:06.002968 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66c51f-0656-49ac-8297-19e448835674 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:06.003247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66c51f-0656-49ac-8297-19e448835674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:06.003534 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66c51f-0656-49ac-8297-19e448835674 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:06.004199 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66c51f-0656-49ac-8297-19e448835674 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:06.006323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66c51f-0656-49ac-8297-19e448835674 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:06.079860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e19d8-a2ff-45e8-96fc-3567fa977d7f service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:06.087306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 935/1834] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:05 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:06.098441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e19d8-a2ff-45e8-96fc-3567fa977d7f service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:06.109072 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863939e2-90c5-432a-9a69-c713bb6088af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:06.109720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863939e2-90c5-432a-9a69-c713bb6088af admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:06.110056 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863939e2-90c5-432a-9a69-c713bb6088af admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:06.110330 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863939e2-90c5-432a-9a69-c713bb6088af admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:06.111032 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863939e2-90c5-432a-9a69-c713bb6088af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:06.112938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863939e2-90c5-432a-9a69-c713bb6088af admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:06.139809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 900/1835] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:06.154579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66c51f-0656-49ac-8297-19e448835674 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:06.171958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66c51f-0656-49ac-8297-19e448835674 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:06.178509 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa18e9-995d-44cb-942d-626e2f8d056d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:06.179138 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa18e9-995d-44cb-942d-626e2f8d056d admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:06.179523 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa18e9-995d-44cb-942d-626e2f8d056d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:06.179842 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa18e9-995d-44cb-942d-626e2f8d056d admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:06.180509 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa18e9-995d-44cb-942d-626e2f8d056d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:06.182576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa18e9-995d-44cb-942d-626e2f8d056d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:06.199333 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 936/1836] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:06.199782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863939e2-90c5-432a-9a69-c713bb6088af admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:06.213087 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863939e2-90c5-432a-9a69-c713bb6088af admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:06.220227 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018e5e5-b1f3-462e-b4ce-362f407849b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:06.221420 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018e5e5-b1f3-462e-b4ce-362f407849b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:06.222077 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018e5e5-b1f3-462e-b4ce-362f407849b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:06.222535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018e5e5-b1f3-462e-b4ce-362f407849b5 admin admin] PATH_INFO: `/v3/projects/44b16ed840a34e7685b967deea2cea68/users/12d244f1a729487f91b41d6e02b34b2b/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:06.259141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018e5e5-b1f3-462e-b4ce-362f407849b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b16ed840a34e7685b967deea2cea68, user_id=12d244f1a729487f91b41d6e02b34b2b, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:06.262269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018e5e5-b1f3-462e-b4ce-362f407849b5 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:06.266230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:06.283130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 901/1837] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:06 2022] PUT /identity/v3/projects/44b16ed840a34e7685b967deea2cea68/users/12d244f1a729487f91b41d6e02b34b2b/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:06.293317 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980df6de-dde4-43f4-a82a-fdb6fde25f54 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:06.293607 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980df6de-dde4-43f4-a82a-fdb6fde25f54 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:06.294155 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980df6de-dde4-43f4-a82a-fdb6fde25f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:06.331136 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-980df6de-dde4-43f4-a82a-fdb6fde25f54 None None] MFA Rules not processed for user `12d244f1a729487f91b41d6e02b34b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:06.332851 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-980df6de-dde4-43f4-a82a-fdb6fde25f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:06.387195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:06.408879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 937/1838] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:06.417536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018e5e5-b1f3-462e-b4ce-362f407849b5 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:06.480235 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018e5e5-b1f3-462e-b4ce-362f407849b5 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:06.484894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088e814-bbb6-43ee-914c-4372822c3a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:06.485598 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088e814-bbb6-43ee-914c-4372822c3a7c admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:06.485890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088e814-bbb6-43ee-914c-4372822c3a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:06.486185 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088e814-bbb6-43ee-914c-4372822c3a7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:06.486827 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088e814-bbb6-43ee-914c-4372822c3a7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:06.488878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088e814-bbb6-43ee-914c-4372822c3a7c admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:06.516143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 902/1839] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:06 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:06.520295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980df6de-dde4-43f4-a82a-fdb6fde25f54 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:06.531697 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980df6de-dde4-43f4-a82a-fdb6fde25f54 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:06.589533 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320d0da-77d4-4f33-8d6a-36a57197bc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:06.590142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320d0da-77d4-4f33-8d6a-36a57197bc9e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:06.590417 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320d0da-77d4-4f33-8d6a-36a57197bc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:06.590686 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320d0da-77d4-4f33-8d6a-36a57197bc9e admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:06.591447 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320d0da-77d4-4f33-8d6a-36a57197bc9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:06.593949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320d0da-77d4-4f33-8d6a-36a57197bc9e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:06.614952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 938/1840] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:06 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:06.618880 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088e814-bbb6-43ee-914c-4372822c3a7c admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:06.629916 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088e814-bbb6-43ee-914c-4372822c3a7c admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:06.634676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9eaae-8d3c-4040-8da7-45167b72815d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:06.635377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9eaae-8d3c-4040-8da7-45167b72815d admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:06.635698 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9eaae-8d3c-4040-8da7-45167b72815d admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:06.635973 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9eaae-8d3c-4040-8da7-45167b72815d admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:06.636638 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9eaae-8d3c-4040-8da7-45167b72815d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:06.638303 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9eaae-8d3c-4040-8da7-45167b72815d admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:06.650614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 903/1841] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:06.654871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320d0da-77d4-4f33-8d6a-36a57197bc9e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:06.665472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320d0da-77d4-4f33-8d6a-36a57197bc9e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:06.670425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83601c1b-6f17-4cdd-820c-942d47b28cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:06.671154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83601c1b-6f17-4cdd-820c-942d47b28cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:06.671504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83601c1b-6f17-4cdd-820c-942d47b28cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:06.671845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83601c1b-6f17-4cdd-820c-942d47b28cf4 admin admin] PATH_INFO: `/v3/projects/0110c836d3664e86a0184638f7c1d058/users/0242a25b0d724419a33c3692168f1e41/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:06.687764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83601c1b-6f17-4cdd-820c-942d47b28cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0110c836d3664e86a0184638f7c1d058, user_id=0242a25b0d724419a33c3692168f1e41, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:06.690447 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83601c1b-6f17-4cdd-820c-942d47b28cf4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:06.706348 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 939/1842] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:06 2022] PUT /identity/v3/projects/0110c836d3664e86a0184638f7c1d058/users/0242a25b0d724419a33c3692168f1e41/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:06.710323 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9eaae-8d3c-4040-8da7-45167b72815d admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:06.722422 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9eaae-8d3c-4040-8da7-45167b72815d admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:06.778171 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4ec1d-e973-4c59-88bd-f4b2b3e6b8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:06.779525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4ec1d-e973-4c59-88bd-f4b2b3e6b8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:06.779849 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4ec1d-e973-4c59-88bd-f4b2b3e6b8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:06.780124 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4ec1d-e973-4c59-88bd-f4b2b3e6b8d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:06.780770 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4ec1d-e973-4c59-88bd-f4b2b3e6b8d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:06.782370 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4ec1d-e973-4c59-88bd-f4b2b3e6b8d5 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:06.792872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 904/1843] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:06.797497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83601c1b-6f17-4cdd-820c-942d47b28cf4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:06.807583 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83601c1b-6f17-4cdd-820c-942d47b28cf4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:06.812271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa01fa-8df4-4707-85f1-731431303a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:06.812929 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa01fa-8df4-4707-85f1-731431303a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:06.813215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa01fa-8df4-4707-85f1-731431303a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:06.813538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa01fa-8df4-4707-85f1-731431303a22 admin admin] PATH_INFO: `/v3/projects/0110c836d3664e86a0184638f7c1d058/users/0242a25b0d724419a33c3692168f1e41/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:06.815402 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa01fa-8df4-4707-85f1-731431303a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0110c836d3664e86a0184638f7c1d058, user_id=0242a25b0d724419a33c3692168f1e41, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:06.818079 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fa01fa-8df4-4707-85f1-731431303a22 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:06.829144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 940/1844] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:06 2022] PUT /identity/v3/projects/0110c836d3664e86a0184638f7c1d058/users/0242a25b0d724419a33c3692168f1e41/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:06.839468 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29d551-66b6-472f-9f18-340e1a65ccc5 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:06.839797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29d551-66b6-472f-9f18-340e1a65ccc5 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:06.840072 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29d551-66b6-472f-9f18-340e1a65ccc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:06.873707 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-8e29d551-66b6-472f-9f18-340e1a65ccc5 None None] MFA Rules not processed for user `0242a25b0d724419a33c3692168f1e41`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:06.875399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-8e29d551-66b6-472f-9f18-340e1a65ccc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:06.941772 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:06.962218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 905/1845] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:06 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:06.973695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa01fa-8df4-4707-85f1-731431303a22 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:06.975376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-20fa01fa-8df4-4707-85f1-731431303a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:07.036514 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa01fa-8df4-4707-85f1-731431303a22 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:07.040113 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b954db3-f4f1-4325-8649-473cf703e137 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:07.040710 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b954db3-f4f1-4325-8649-473cf703e137 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:07.040989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b954db3-f4f1-4325-8649-473cf703e137 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:07.041277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b954db3-f4f1-4325-8649-473cf703e137 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:07.050703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b954db3-f4f1-4325-8649-473cf703e137 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:07.052180 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b954db3-f4f1-4325-8649-473cf703e137 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:07.062683 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 941/1846] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:06 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:07.092142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29d551-66b6-472f-9f18-340e1a65ccc5 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:07.181536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29d551-66b6-472f-9f18-340e1a65ccc5 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:07.188511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415c585-7a83-419a-858c-05676ef22650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:07.189757 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415c585-7a83-419a-858c-05676ef22650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:07.189990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415c585-7a83-419a-858c-05676ef22650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:07.191034 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415c585-7a83-419a-858c-05676ef22650 admin admin] PATH_INFO: `/v3/users/12d244f1a729487f91b41d6e02b34b2b` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:07.193689 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9415c585-7a83-419a-858c-05676ef22650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12d244f1a729487f91b41d6e02b34b2b)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:07.198195 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9415c585-7a83-419a-858c-05676ef22650 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:07.257590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-9415c585-7a83-419a-858c-05676ef22650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12d244f1a729487f91b41d6e02b34b2b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:07.258327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-9415c585-7a83-419a-858c-05676ef22650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12d244f1a729487f91b41d6e02b34b2b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:07.259132 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-9415c585-7a83-419a-858c-05676ef22650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12d244f1a729487f91b41d6e02b34b2b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:07.266879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-9415c585-7a83-419a-858c-05676ef22650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12d244f1a729487f91b41d6e02b34b2b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:07.275903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-9415c585-7a83-419a-858c-05676ef22650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12d244f1a729487f91b41d6e02b34b2b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:07.309172 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 906/1847] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:07 2022] DELETE /identity/v3/users/12d244f1a729487f91b41d6e02b34b2b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:07.314461 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb30796-7504-4e31-90ad-ffd348669d58 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:07.314815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb30796-7504-4e31-90ad-ffd348669d58 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:07.315157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb30796-7504-4e31-90ad-ffd348669d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:07.337292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1bb30796-7504-4e31-90ad-ffd348669d58 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:07.338527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1bb30796-7504-4e31-90ad-ffd348669d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:07.406235 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415c585-7a83-419a-858c-05676ef22650 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:07.406871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:07.407862 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-9415c585-7a83-419a-858c-05676ef22650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:07.428608 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 942/1848] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:07.438627 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb30796-7504-4e31-90ad-ffd348669d58 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:07.440759 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1bb30796-7504-4e31-90ad-ffd348669d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:07.478470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415c585-7a83-419a-858c-05676ef22650 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:07.482628 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56251899-326d-4231-9e30-83ced7a37187 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:07.483380 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56251899-326d-4231-9e30-83ced7a37187 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:07.483692 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56251899-326d-4231-9e30-83ced7a37187 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:07.483969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56251899-326d-4231-9e30-83ced7a37187 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:07.485713 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-56251899-326d-4231-9e30-83ced7a37187 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:07.500343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb30796-7504-4e31-90ad-ffd348669d58 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:07.504036 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089e78d-4fc8-4211-8ba7-07f860786f07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:07.504647 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089e78d-4fc8-4211-8ba7-07f860786f07 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:07.504928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089e78d-4fc8-4211-8ba7-07f860786f07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:07.505205 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089e78d-4fc8-4211-8ba7-07f860786f07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:07.514348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:07.514520 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089e78d-4fc8-4211-8ba7-07f860786f07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:07.516002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9089e78d-4fc8-4211-8ba7-07f860786f07 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:07.526108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 943/1849] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:07.556873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56251899-326d-4231-9e30-83ced7a37187 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:07.558315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56251899-326d-4231-9e30-83ced7a37187 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:07.568286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 907/1850] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:07.804479 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089e78d-4fc8-4211-8ba7-07f860786f07 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:07.806089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9089e78d-4fc8-4211-8ba7-07f860786f07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:07.823159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089e78d-4fc8-4211-8ba7-07f860786f07 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:07.831388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f00c0f-bac7-43f6-90ef-88ec851c1594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:07.831388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f00c0f-bac7-43f6-90ef-88ec851c1594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:07.831680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f00c0f-bac7-43f6-90ef-88ec851c1594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:07.831772 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f00c0f-bac7-43f6-90ef-88ec851c1594 admin admin] PATH_INFO: `/v3/projects/44b16ed840a34e7685b967deea2cea68` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:07.832993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f00c0f-bac7-43f6-90ef-88ec851c1594 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44b16ed840a34e7685b967deea2cea68)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:07.835321 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f00c0f-bac7-43f6-90ef-88ec851c1594 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:07.889071 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c6f00c0f-bac7-43f6-90ef-88ec851c1594 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44b16ed840a34e7685b967deea2cea68', 'request_id': 'req-c6f00c0f-bac7-43f6-90ef-88ec851c1594'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:07.894939 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c6f00c0f-bac7-43f6-90ef-88ec851c1594 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44b16ed840a34e7685b967deea2cea68', 'request_id': 'req-c6f00c0f-bac7-43f6-90ef-88ec851c1594'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:07.905069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c6f00c0f-bac7-43f6-90ef-88ec851c1594 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44b16ed840a34e7685b967deea2cea68', 'request_id': 'req-c6f00c0f-bac7-43f6-90ef-88ec851c1594'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:07.924080 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c6f00c0f-bac7-43f6-90ef-88ec851c1594 admin admin] The token cache is being invalidate because project 44b16ed840a34e7685b967deea2cea68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:07.924694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c6f00c0f-bac7-43f6-90ef-88ec851c1594 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:07.926123 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c6f00c0f-bac7-43f6-90ef-88ec851c1594 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:07.928215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 944/1851] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:07 2022] DELETE /identity/v3/projects/44b16ed840a34e7685b967deea2cea68 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:07.932423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56251899-326d-4231-9e30-83ced7a37187 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:07.943489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56251899-326d-4231-9e30-83ced7a37187 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:08.008359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cc534-840a-4b60-b7f9-67df04171748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:08.009040 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cc534-840a-4b60-b7f9-67df04171748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:08.009343 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cc534-840a-4b60-b7f9-67df04171748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:08.009603 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cc534-840a-4b60-b7f9-67df04171748 admin admin] PATH_INFO: `/v3/users/0242a25b0d724419a33c3692168f1e41` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:08.010972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0cc534-840a-4b60-b7f9-67df04171748 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0242a25b0d724419a33c3692168f1e41)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:08.012964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0cc534-840a-4b60-b7f9-67df04171748 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:08.062009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3d0cc534-840a-4b60-b7f9-67df04171748 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0242a25b0d724419a33c3692168f1e41'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:08.062670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3d0cc534-840a-4b60-b7f9-67df04171748 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0242a25b0d724419a33c3692168f1e41'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:08.063289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3d0cc534-840a-4b60-b7f9-67df04171748 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0242a25b0d724419a33c3692168f1e41'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:08.070759 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3d0cc534-840a-4b60-b7f9-67df04171748 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0242a25b0d724419a33c3692168f1e41'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:08.080574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3d0cc534-840a-4b60-b7f9-67df04171748 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0242a25b0d724419a33c3692168f1e41'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:08.108293 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 908/1852] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:07 2022] DELETE /identity/v3/users/0242a25b0d724419a33c3692168f1e41 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:08.617940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f00c0f-bac7-43f6-90ef-88ec851c1594 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:08.619851 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c6f00c0f-bac7-43f6-90ef-88ec851c1594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:08.637815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f00c0f-bac7-43f6-90ef-88ec851c1594 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:08.720842 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4d3f7-288f-4062-b8a6-68b948b56afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:08.721588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4d3f7-288f-4062-b8a6-68b948b56afb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:08.721934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4d3f7-288f-4062-b8a6-68b948b56afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:08.722275 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4d3f7-288f-4062-b8a6-68b948b56afb admin admin] PATH_INFO: `/v3/projects/0110c836d3664e86a0184638f7c1d058` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:08.723572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4d3f7-288f-4062-b8a6-68b948b56afb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0110c836d3664e86a0184638f7c1d058)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:08.726061 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4d3f7-288f-4062-b8a6-68b948b56afb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:08.750838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:08.793774 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2ab4d3f7-288f-4062-b8a6-68b948b56afb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0110c836d3664e86a0184638f7c1d058', 'request_id': 'req-2ab4d3f7-288f-4062-b8a6-68b948b56afb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:08.801836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2ab4d3f7-288f-4062-b8a6-68b948b56afb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0110c836d3664e86a0184638f7c1d058', 'request_id': 'req-2ab4d3f7-288f-4062-b8a6-68b948b56afb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:08.812566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2ab4d3f7-288f-4062-b8a6-68b948b56afb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0110c836d3664e86a0184638f7c1d058', 'request_id': 'req-2ab4d3f7-288f-4062-b8a6-68b948b56afb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:08.830282 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2ab4d3f7-288f-4062-b8a6-68b948b56afb admin admin] The token cache is being invalidate because project 0110c836d3664e86a0184638f7c1d058 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:08.830651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2ab4d3f7-288f-4062-b8a6-68b948b56afb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:08.832240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2ab4d3f7-288f-4062-b8a6-68b948b56afb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:08.837382 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 945/1853] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:08 2022] DELETE /identity/v3/projects/0110c836d3664e86a0184638f7c1d058 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:08.841078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75406e-7386-4f1c-8b6c-5e3f8608cba6 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:08.842479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75406e-7386-4f1c-8b6c-5e3f8608cba6 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:08.843555 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75406e-7386-4f1c-8b6c-5e3f8608cba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:08.867226 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-2a75406e-7386-4f1c-8b6c-5e3f8608cba6 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:08.868531 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2a75406e-7386-4f1c-8b6c-5e3f8608cba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:08.918902 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:08.975427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4471dae-f5ec-44bc-964a-72a5044ff9d4 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:08.975797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4471dae-f5ec-44bc-964a-72a5044ff9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:08.976174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4471dae-f5ec-44bc-964a-72a5044ff9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:08.981109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 909/1854] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:09.010906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b4471dae-f5ec-44bc-964a-72a5044ff9d4 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:09.012664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b4471dae-f5ec-44bc-964a-72a5044ff9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:09.027932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 946/1855] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:09.032796 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75406e-7386-4f1c-8b6c-5e3f8608cba6 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:09.049144 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75406e-7386-4f1c-8b6c-5e3f8608cba6 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:09.061227 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17fd26-2d0e-40d1-826b-48dde8e42995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:09.062169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17fd26-2d0e-40d1-826b-48dde8e42995 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:09.062620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17fd26-2d0e-40d1-826b-48dde8e42995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:09.063045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa17fd26-2d0e-40d1-826b-48dde8e42995 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:09.064103 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa17fd26-2d0e-40d1-826b-48dde8e42995 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:09.064495 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa17fd26-2d0e-40d1-826b-48dde8e42995 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:09.066403 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa17fd26-2d0e-40d1-826b-48dde8e42995 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:09.087508 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 910/1856] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:09.089110 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4471dae-f5ec-44bc-964a-72a5044ff9d4 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:09.101869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4471dae-f5ec-44bc-964a-72a5044ff9d4 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:09.109712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198b339-5bb0-4948-80bf-f233a761e080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:09.110488 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198b339-5bb0-4948-80bf-f233a761e080 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:09.110873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198b339-5bb0-4948-80bf-f233a761e080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:09.111248 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198b339-5bb0-4948-80bf-f233a761e080 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:09.112145 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198b339-5bb0-4948-80bf-f233a761e080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:09.114664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198b339-5bb0-4948-80bf-f233a761e080 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:09.175685 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 947/1857] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:09 2022] POST /identity/v3/projects => generated 381 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:09.180976 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17fd26-2d0e-40d1-826b-48dde8e42995 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:09.192803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa17fd26-2d0e-40d1-826b-48dde8e42995 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:09.313952 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7e5a1-d2f5-4196-ac55-f35c9d69383e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:09.314626 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7e5a1-d2f5-4196-ac55-f35c9d69383e admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:09.314972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7e5a1-d2f5-4196-ac55-f35c9d69383e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:09.315314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7e5a1-d2f5-4196-ac55-f35c9d69383e admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:09.316012 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d7e5a1-d2f5-4196-ac55-f35c9d69383e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:09.318113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d7e5a1-d2f5-4196-ac55-f35c9d69383e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:09.349764 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 911/1858] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:09 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:09.350967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198b339-5bb0-4948-80bf-f233a761e080 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:09.363107 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198b339-5bb0-4948-80bf-f233a761e080 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:09.368797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fba448-6f0e-4b48-a197-9ccb318c9af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:09.369501 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fba448-6f0e-4b48-a197-9ccb318c9af7 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:09.370209 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fba448-6f0e-4b48-a197-9ccb318c9af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:09.370209 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fba448-6f0e-4b48-a197-9ccb318c9af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:09.370874 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fba448-6f0e-4b48-a197-9ccb318c9af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:09.373443 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fba448-6f0e-4b48-a197-9ccb318c9af7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:09.387736 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 948/1859] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:09.390424 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7e5a1-d2f5-4196-ac55-f35c9d69383e admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:09.418700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7e5a1-d2f5-4196-ac55-f35c9d69383e admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:09.435898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabd60b-7035-4d5f-91a8-8f075e6a5b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:09.435898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabd60b-7035-4d5f-91a8-8f075e6a5b3d admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:09.435898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabd60b-7035-4d5f-91a8-8f075e6a5b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:09.435898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabd60b-7035-4d5f-91a8-8f075e6a5b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:09.436656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dabd60b-7035-4d5f-91a8-8f075e6a5b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:09.436656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dabd60b-7035-4d5f-91a8-8f075e6a5b3d admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:09.445216 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 912/1860] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:09.450376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fba448-6f0e-4b48-a197-9ccb318c9af7 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:09.464950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fba448-6f0e-4b48-a197-9ccb318c9af7 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:09.471323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499813db-3592-4fc8-b013-e311efaca68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:09.472155 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499813db-3592-4fc8-b013-e311efaca68c admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:09.472512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499813db-3592-4fc8-b013-e311efaca68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:09.473010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499813db-3592-4fc8-b013-e311efaca68c admin admin] PATH_INFO: `/v3/projects/982711184b9349dfa4bba42e5c207b38/users/69488b80c21f4e318e2a6dae0f6e97d4/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:09.494420 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499813db-3592-4fc8-b013-e311efaca68c admin admin] RBAC: Authorizing `identity:create_grant(project_id=982711184b9349dfa4bba42e5c207b38, user_id=69488b80c21f4e318e2a6dae0f6e97d4, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:09.497724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499813db-3592-4fc8-b013-e311efaca68c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:09.515453 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 949/1861] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:09 2022] PUT /identity/v3/projects/982711184b9349dfa4bba42e5c207b38/users/69488b80c21f4e318e2a6dae0f6e97d4/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:09.526026 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32ceb5-61fb-45af-bebc-6f5f2f77e488 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:09.526796 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32ceb5-61fb-45af-bebc-6f5f2f77e488 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:09.527194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32ceb5-61fb-45af-bebc-6f5f2f77e488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:09.605242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-3e32ceb5-61fb-45af-bebc-6f5f2f77e488 None None] MFA Rules not processed for user `69488b80c21f4e318e2a6dae0f6e97d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:09.607462 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-3e32ceb5-61fb-45af-bebc-6f5f2f77e488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:09.702664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 913/1862] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:09.707567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499813db-3592-4fc8-b013-e311efaca68c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:09.838291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499813db-3592-4fc8-b013-e311efaca68c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:09.861248 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d97d65-ce40-4635-b5fc-00f43ec84b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:09.861883 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d97d65-ce40-4635-b5fc-00f43ec84b79 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:09.862173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d97d65-ce40-4635-b5fc-00f43ec84b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:09.862451 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d97d65-ce40-4635-b5fc-00f43ec84b79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:09.863189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d97d65-ce40-4635-b5fc-00f43ec84b79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:09.866519 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d97d65-ce40-4635-b5fc-00f43ec84b79 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:09.904940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 950/1863] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:09 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:09.906276 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32ceb5-61fb-45af-bebc-6f5f2f77e488 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:09.922274 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32ceb5-61fb-45af-bebc-6f5f2f77e488 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:09.981453 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:10.020507 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e99d68-cbb7-460a-a8f1-59df1d57f4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:10.021857 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e99d68-cbb7-460a-a8f1-59df1d57f4a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:10.022247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e99d68-cbb7-460a-a8f1-59df1d57f4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:10.022562 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e99d68-cbb7-460a-a8f1-59df1d57f4a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:10.031084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e99d68-cbb7-460a-a8f1-59df1d57f4a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:10.034698 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e99d68-cbb7-460a-a8f1-59df1d57f4a1 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:10.071824 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 914/1864] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:09 2022] POST /identity/v3/users => generated 296 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:10.074207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d97d65-ce40-4635-b5fc-00f43ec84b79 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:10.089795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d97d65-ce40-4635-b5fc-00f43ec84b79 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:10.097307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644872a-9cd9-4206-a108-2d9ca51c0564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:10.098314 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644872a-9cd9-4206-a108-2d9ca51c0564 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:10.098625 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644872a-9cd9-4206-a108-2d9ca51c0564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:10.099259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644872a-9cd9-4206-a108-2d9ca51c0564 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:10.100146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644872a-9cd9-4206-a108-2d9ca51c0564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:10.101910 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644872a-9cd9-4206-a108-2d9ca51c0564 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:10.118092 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 951/1865] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:10.119440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e99d68-cbb7-460a-a8f1-59df1d57f4a1 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:10.146423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e99d68-cbb7-460a-a8f1-59df1d57f4a1 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:10.168147 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4dcc4b-5f4f-44e1-901e-119b18c4cb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:10.168967 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dcc4b-5f4f-44e1-901e-119b18c4cb54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:10.169921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dcc4b-5f4f-44e1-901e-119b18c4cb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:10.170328 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4dcc4b-5f4f-44e1-901e-119b18c4cb54 admin admin] PATH_INFO: `/v3/projects/b63b3265baf14f0692e924211759a4b3/users/63cc556f1baf4348bb39aadaa9222635/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:10.208248 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4dcc4b-5f4f-44e1-901e-119b18c4cb54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63b3265baf14f0692e924211759a4b3, user_id=63cc556f1baf4348bb39aadaa9222635, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:10.213270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4dcc4b-5f4f-44e1-901e-119b18c4cb54 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:10.258307 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 915/1866] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:10 2022] PUT /identity/v3/projects/b63b3265baf14f0692e924211759a4b3/users/63cc556f1baf4348bb39aadaa9222635/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:10.262565 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644872a-9cd9-4206-a108-2d9ca51c0564 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:10.277197 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644872a-9cd9-4206-a108-2d9ca51c0564 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:10.296311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:10.411345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a92a7-10ed-4b31-b667-e088420f2841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:10.412415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a92a7-10ed-4b31-b667-e088420f2841 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:10.412834 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a92a7-10ed-4b31-b667-e088420f2841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:10.413227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a92a7-10ed-4b31-b667-e088420f2841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:10.417421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a92a7-10ed-4b31-b667-e088420f2841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:10.417851 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a92a7-10ed-4b31-b667-e088420f2841 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:10.439658 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 952/1867] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:10.442803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4dcc4b-5f4f-44e1-901e-119b18c4cb54 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:10.465642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4dcc4b-5f4f-44e1-901e-119b18c4cb54 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:10.495945 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abdb927-f984-4268-8ef5-996ec27f02c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:10.496099 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdb927-f984-4268-8ef5-996ec27f02c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:10.496572 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdb927-f984-4268-8ef5-996ec27f02c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:10.497003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abdb927-f984-4268-8ef5-996ec27f02c4 admin admin] PATH_INFO: `/v3/projects/b63b3265baf14f0692e924211759a4b3/users/63cc556f1baf4348bb39aadaa9222635/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:10.501627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abdb927-f984-4268-8ef5-996ec27f02c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63b3265baf14f0692e924211759a4b3, user_id=63cc556f1baf4348bb39aadaa9222635, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:10.506985 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abdb927-f984-4268-8ef5-996ec27f02c4 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:10.535225 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 916/1868] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:10 2022] PUT /identity/v3/projects/b63b3265baf14f0692e924211759a4b3/users/63cc556f1baf4348bb39aadaa9222635/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:10.537844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6b245-51ca-487b-afa1-9c4366ccae9c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:10.538392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6b245-51ca-487b-afa1-9c4366ccae9c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:10.539142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6b245-51ca-487b-afa1-9c4366ccae9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:10.609982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-26d6b245-51ca-487b-afa1-9c4366ccae9c None None] MFA Rules not processed for user `63cc556f1baf4348bb39aadaa9222635`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:10.611439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-26d6b245-51ca-487b-afa1-9c4366ccae9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:10.719536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 953/1869] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:10.733622 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abdb927-f984-4268-8ef5-996ec27f02c4 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:10.735828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-9abdb927-f984-4268-8ef5-996ec27f02c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:10.846924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abdb927-f984-4268-8ef5-996ec27f02c4 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:10.850812 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5599e97-1aa2-40f9-9a1c-b16bc0af075e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:10.851474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5599e97-1aa2-40f9-9a1c-b16bc0af075e service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:10.851801 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5599e97-1aa2-40f9-9a1c-b16bc0af075e service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:10.852113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5599e97-1aa2-40f9-9a1c-b16bc0af075e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:10.890768 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5599e97-1aa2-40f9-9a1c-b16bc0af075e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:10.907903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5599e97-1aa2-40f9-9a1c-b16bc0af075e service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:10.950045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 917/1870] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:10 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:11.785034 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6b245-51ca-487b-afa1-9c4366ccae9c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:11.941069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6b245-51ca-487b-afa1-9c4366ccae9c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:11.959207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524ca59-b998-4fe3-ac73-07ade0c6ef11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:11.959905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524ca59-b998-4fe3-ac73-07ade0c6ef11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:11.960357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524ca59-b998-4fe3-ac73-07ade0c6ef11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:11.960836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524ca59-b998-4fe3-ac73-07ade0c6ef11 admin admin] PATH_INFO: `/v3/users/69488b80c21f4e318e2a6dae0f6e97d4` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:11.963434 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524ca59-b998-4fe3-ac73-07ade0c6ef11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69488b80c21f4e318e2a6dae0f6e97d4)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:11.970487 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524ca59-b998-4fe3-ac73-07ade0c6ef11 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:11.996395 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5599e97-1aa2-40f9-9a1c-b16bc0af075e service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:12.003895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d5599e97-1aa2-40f9-9a1c-b16bc0af075e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:12.035191 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5599e97-1aa2-40f9-9a1c-b16bc0af075e service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:12.050008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876fd78-6ff5-4083-b481-7e8bf1206b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:12.052422 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876fd78-6ff5-4083-b481-7e8bf1206b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:12.053930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876fd78-6ff5-4083-b481-7e8bf1206b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:12.056704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876fd78-6ff5-4083-b481-7e8bf1206b25 admin admin] PATH_INFO: `/v3/users/2c0edaaff9f543e6955df2e07e86054d` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:12.066144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:12.071978 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876fd78-6ff5-4083-b481-7e8bf1206b25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0edaaff9f543e6955df2e07e86054d)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:12.074127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876fd78-6ff5-4083-b481-7e8bf1206b25 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:12.088845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8524ca59-b998-4fe3-ac73-07ade0c6ef11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69488b80c21f4e318e2a6dae0f6e97d4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.090684 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8524ca59-b998-4fe3-ac73-07ade0c6ef11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69488b80c21f4e318e2a6dae0f6e97d4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.114446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8524ca59-b998-4fe3-ac73-07ade0c6ef11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69488b80c21f4e318e2a6dae0f6e97d4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.125908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8524ca59-b998-4fe3-ac73-07ade0c6ef11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69488b80c21f4e318e2a6dae0f6e97d4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.156315 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8524ca59-b998-4fe3-ac73-07ade0c6ef11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69488b80c21f4e318e2a6dae0f6e97d4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.169635 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 954/1871] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:11 2022] DELETE /identity/v3/users/69488b80c21f4e318e2a6dae0f6e97d4 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:12.174096 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ab646-ac8e-46f9-aac2-d20847d74a64 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:12.174378 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ab646-ac8e-46f9-aac2-d20847d74a64 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:12.174659 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ab646-ac8e-46f9-aac2-d20847d74a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:12.182553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d876fd78-6ff5-4083-b481-7e8bf1206b25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0edaaff9f543e6955df2e07e86054d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.185403 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d876fd78-6ff5-4083-b481-7e8bf1206b25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0edaaff9f543e6955df2e07e86054d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.186264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d876fd78-6ff5-4083-b481-7e8bf1206b25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0edaaff9f543e6955df2e07e86054d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.199550 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d876fd78-6ff5-4083-b481-7e8bf1206b25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0edaaff9f543e6955df2e07e86054d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.224168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-098ab646-ac8e-46f9-aac2-d20847d74a64 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:12.225407 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-098ab646-ac8e-46f9-aac2-d20847d74a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:12.232454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d876fd78-6ff5-4083-b481-7e8bf1206b25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0edaaff9f543e6955df2e07e86054d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:12.274934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 918/1872] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:11 2022] DELETE /identity/v3/users/2c0edaaff9f543e6955df2e07e86054d => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:12.284563 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3f471-c0f3-4116-8bd9-8a210d55326b None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:12.285582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3f471-c0f3-4116-8bd9-8a210d55326b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:12.287912 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3f471-c0f3-4116-8bd9-8a210d55326b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:12.304821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:12.334588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 955/1873] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:12.339327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-85c3f471-c0f3-4116-8bd9-8a210d55326b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:12.342312 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-85c3f471-c0f3-4116-8bd9-8a210d55326b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:12.349197 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ab646-ac8e-46f9-aac2-d20847d74a64 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:12.350874 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-098ab646-ac8e-46f9-aac2-d20847d74a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:12.425527 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 919/1874] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:12.430191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ab646-ac8e-46f9-aac2-d20847d74a64 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:12.440504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3f471-c0f3-4116-8bd9-8a210d55326b None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:12.495227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88542fd-6e3f-4818-bd64-c51d9d9c90df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:12.495878 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88542fd-6e3f-4818-bd64-c51d9d9c90df service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:12.496156 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88542fd-6e3f-4818-bd64-c51d9d9c90df service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:12.496426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88542fd-6e3f-4818-bd64-c51d9d9c90df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:12.508308 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88542fd-6e3f-4818-bd64-c51d9d9c90df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:12.509853 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88542fd-6e3f-4818-bd64-c51d9d9c90df service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:12.523134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 956/1875] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:12.526322 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3f471-c0f3-4116-8bd9-8a210d55326b None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:12.530921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ee517-cb00-4c7a-acf7-691368c8e7a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:12.531544 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ee517-cb00-4c7a-acf7-691368c8e7a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:12.531850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ee517-cb00-4c7a-acf7-691368c8e7a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:12.532127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ee517-cb00-4c7a-acf7-691368c8e7a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:12.546937 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ee517-cb00-4c7a-acf7-691368c8e7a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:12.548439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ee517-cb00-4c7a-acf7-691368c8e7a6 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:12.558755 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 920/1876] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:13.474660 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88542fd-6e3f-4818-bd64-c51d9d9c90df service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:13.478122 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d88542fd-6e3f-4818-bd64-c51d9d9c90df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:13.498803 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88542fd-6e3f-4818-bd64-c51d9d9c90df service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:13.518086 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:13.518765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:13.519051 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:13.519472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1 admin admin] PATH_INFO: `/v3/projects/982711184b9349dfa4bba42e5c207b38` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:13.520822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=982711184b9349dfa4bba42e5c207b38)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:13.524242 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:13.538057 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:13.629820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '982711184b9349dfa4bba42e5c207b38', 'request_id': 'req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.636978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '982711184b9349dfa4bba42e5c207b38', 'request_id': 'req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.647798 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '982711184b9349dfa4bba42e5c207b38', 'request_id': 'req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.661962 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1 admin admin] The token cache is being invalidate because project 982711184b9349dfa4bba42e5c207b38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:13.662373 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.663147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.666023 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 957/1877] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:13 2022] DELETE /identity/v3/projects/982711184b9349dfa4bba42e5c207b38 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:13.669901 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ee517-cb00-4c7a-acf7-691368c8e7a6 service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:13.682899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ee517-cb00-4c7a-acf7-691368c8e7a6 service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:13.746623 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:13.749177 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:13.764658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be62e8a-dcb5-4c05-a4ba-7b56281d0d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:13.765238 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be62e8a-dcb5-4c05-a4ba-7b56281d0d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:13.765745 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be62e8a-dcb5-4c05-a4ba-7b56281d0d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:13.766828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be62e8a-dcb5-4c05-a4ba-7b56281d0d22 admin admin] PATH_INFO: `/v3/users/63cc556f1baf4348bb39aadaa9222635` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:13.769269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be62e8a-dcb5-4c05-a4ba-7b56281d0d22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63cc556f1baf4348bb39aadaa9222635)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:13.771962 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be62e8a-dcb5-4c05-a4ba-7b56281d0d22 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:13.860062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b9fd1-303a-4fab-bbff-0b0d08f386f1 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:13.865652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cd836-88fe-4c76-972b-f9571c521922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:13.866640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cd836-88fe-4c76-972b-f9571c521922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:13.867065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cd836-88fe-4c76-972b-f9571c521922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:13.867651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cd836-88fe-4c76-972b-f9571c521922 admin admin] PATH_INFO: `/v3/projects/156ce21af05e4b378a5851367cb7d508` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:13.868110 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3be62e8a-dcb5-4c05-a4ba-7b56281d0d22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63cc556f1baf4348bb39aadaa9222635'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.869701 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cd836-88fe-4c76-972b-f9571c521922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=156ce21af05e4b378a5851367cb7d508)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:13.870176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3be62e8a-dcb5-4c05-a4ba-7b56281d0d22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63cc556f1baf4348bb39aadaa9222635'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.871104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3be62e8a-dcb5-4c05-a4ba-7b56281d0d22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63cc556f1baf4348bb39aadaa9222635'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.872868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cd836-88fe-4c76-972b-f9571c521922 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:13.873589 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:13.892061 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3be62e8a-dcb5-4c05-a4ba-7b56281d0d22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63cc556f1baf4348bb39aadaa9222635'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.907041 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3be62e8a-dcb5-4c05-a4ba-7b56281d0d22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63cc556f1baf4348bb39aadaa9222635'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.932236 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 921/1878] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:13 2022] DELETE /identity/v3/users/63cc556f1baf4348bb39aadaa9222635 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:13.934017 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70806fd0-bcc8-463b-a5a7-32f38abf0632 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:13.934313 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70806fd0-bcc8-463b-a5a7-32f38abf0632 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:13.942130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-558cd836-88fe-4c76-972b-f9571c521922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '156ce21af05e4b378a5851367cb7d508', 'request_id': 'req-558cd836-88fe-4c76-972b-f9571c521922'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.944313 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70806fd0-bcc8-463b-a5a7-32f38abf0632 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:13.949122 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-558cd836-88fe-4c76-972b-f9571c521922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '156ce21af05e4b378a5851367cb7d508', 'request_id': 'req-558cd836-88fe-4c76-972b-f9571c521922'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.950150 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 922/1879] 10.222.0.58 () {54 vars in 921 bytes} [Wed Aug 24 12:33:13 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:33:13.959254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-558cd836-88fe-4c76-972b-f9571c521922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '156ce21af05e4b378a5851367cb7d508', 'request_id': 'req-558cd836-88fe-4c76-972b-f9571c521922'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.965230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7865d1-e0c3-43ed-95a7-5d948a59d42f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:13.965523 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7865d1-e0c3-43ed-95a7-5d948a59d42f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:13.965803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7865d1-e0c3-43ed-95a7-5d948a59d42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:13.979464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-558cd836-88fe-4c76-972b-f9571c521922 admin admin] The token cache is being invalidate because project 156ce21af05e4b378a5851367cb7d508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:13.979831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-558cd836-88fe-4c76-972b-f9571c521922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.980800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-558cd836-88fe-4c76-972b-f9571c521922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:13.987037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 958/1880] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:13 2022] DELETE /identity/v3/projects/156ce21af05e4b378a5851367cb7d508 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:13.989551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cd836-88fe-4c76-972b-f9571c521922 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:13.991322 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-558cd836-88fe-4c76-972b-f9571c521922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:13.996427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-9b7865d1-e0c3-43ed-95a7-5d948a59d42f None None] MFA Rules not processed for user `311dbed3ab0b44a7830a93227ff9ad33`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:13.998057 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-9b7865d1-e0c3-43ed-95a7-5d948a59d42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:14.035075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:14.083333 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cd836-88fe-4c76-972b-f9571c521922 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:14.170682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f43b6-af71-49be-9adf-1aa3a4a3b0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:14.171145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 923/1881] 10.222.0.58 () {58 vars in 1018 bytes} [Wed Aug 24 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:14.171587 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f43b6-af71-49be-9adf-1aa3a4a3b0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:14.172069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f43b6-af71-49be-9adf-1aa3a4a3b0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:14.172439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f43b6-af71-49be-9adf-1aa3a4a3b0d3 admin admin] PATH_INFO: `/v3/users/adb994037a454bc5b9ef5931077104e4` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:14.174241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f43b6-af71-49be-9adf-1aa3a4a3b0d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adb994037a454bc5b9ef5931077104e4)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:14.176443 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9f43b6-af71-49be-9adf-1aa3a4a3b0d3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:14.261925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-be9f43b6-af71-49be-9adf-1aa3a4a3b0d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adb994037a454bc5b9ef5931077104e4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:14.271125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-be9f43b6-af71-49be-9adf-1aa3a4a3b0d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adb994037a454bc5b9ef5931077104e4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:14.311457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-be9f43b6-af71-49be-9adf-1aa3a4a3b0d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adb994037a454bc5b9ef5931077104e4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:14.335280 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-be9f43b6-af71-49be-9adf-1aa3a4a3b0d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adb994037a454bc5b9ef5931077104e4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:14.380357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-be9f43b6-af71-49be-9adf-1aa3a4a3b0d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adb994037a454bc5b9ef5931077104e4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:14.411856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 959/1882] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:13 2022] DELETE /identity/v3/users/adb994037a454bc5b9ef5931077104e4 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:14.420080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7865d1-e0c3-43ed-95a7-5d948a59d42f None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:14.421571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-9b7865d1-e0c3-43ed-95a7-5d948a59d42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:14.459991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7865d1-e0c3-43ed-95a7-5d948a59d42f None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:14.634544 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c71db-86d8-4df1-8606-3d9e55312734 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:14.635151 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c71db-86d8-4df1-8606-3d9e55312734 service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:14.635663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c71db-86d8-4df1-8606-3d9e55312734 service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:14.635857 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c71db-86d8-4df1-8606-3d9e55312734 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:14.639085 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-777c71db-86d8-4df1-8606-3d9e55312734 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:14.680285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c71db-86d8-4df1-8606-3d9e55312734 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:14.683906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777c71db-86d8-4df1-8606-3d9e55312734 service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:14.699874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 924/1883] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:14.832055 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f43b6-af71-49be-9adf-1aa3a4a3b0d3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:14.833553 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-be9f43b6-af71-49be-9adf-1aa3a4a3b0d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:14.918098 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:14.941311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9f43b6-af71-49be-9adf-1aa3a4a3b0d3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:14.947351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81701f-41c6-40a0-b9e9-3392384bdd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:14.948489 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81701f-41c6-40a0-b9e9-3392384bdd29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:14.948489 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81701f-41c6-40a0-b9e9-3392384bdd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:14.948837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81701f-41c6-40a0-b9e9-3392384bdd29 admin admin] PATH_INFO: `/v3/projects/b63b3265baf14f0692e924211759a4b3` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:14.950034 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81701f-41c6-40a0-b9e9-3392384bdd29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b63b3265baf14f0692e924211759a4b3)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:14.953228 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e81701f-41c6-40a0-b9e9-3392384bdd29 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:15.018302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9e81701f-41c6-40a0-b9e9-3392384bdd29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b63b3265baf14f0692e924211759a4b3', 'request_id': 'req-9e81701f-41c6-40a0-b9e9-3392384bdd29'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.028456 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9e81701f-41c6-40a0-b9e9-3392384bdd29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b63b3265baf14f0692e924211759a4b3', 'request_id': 'req-9e81701f-41c6-40a0-b9e9-3392384bdd29'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.040973 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9e81701f-41c6-40a0-b9e9-3392384bdd29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b63b3265baf14f0692e924211759a4b3', 'request_id': 'req-9e81701f-41c6-40a0-b9e9-3392384bdd29'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.056377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9e81701f-41c6-40a0-b9e9-3392384bdd29 admin admin] The token cache is being invalidate because project b63b3265baf14f0692e924211759a4b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:15.056819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9e81701f-41c6-40a0-b9e9-3392384bdd29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.057963 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9e81701f-41c6-40a0-b9e9-3392384bdd29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.059846 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 960/1884] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:14 2022] DELETE /identity/v3/projects/b63b3265baf14f0692e924211759a4b3 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:15.064567 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c629b-b687-404d-b4e9-1ac560593b33 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.065051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c629b-b687-404d-b4e9-1ac560593b33 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.065479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c629b-b687-404d-b4e9-1ac560593b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.085635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-f36c629b-b687-404d-b4e9-1ac560593b33 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:15.086844 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f36c629b-b687-404d-b4e9-1ac560593b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:15.147660 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:15.158083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81701f-41c6-40a0-b9e9-3392384bdd29 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:15.160009 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9e81701f-41c6-40a0-b9e9-3392384bdd29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:15.183882 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 925/1885] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:15.191890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4718210-8dcb-4b16-9681-6b8819c2b863 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.192180 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4718210-8dcb-4b16-9681-6b8819c2b863 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.192457 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4718210-8dcb-4b16-9681-6b8819c2b863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.220896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-a4718210-8dcb-4b16-9681-6b8819c2b863 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:15.222577 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a4718210-8dcb-4b16-9681-6b8819c2b863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:15.238470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 926/1886] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:15.240627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4718210-8dcb-4b16-9681-6b8819c2b863 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:15.253079 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81701f-41c6-40a0-b9e9-3392384bdd29 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:15.265561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccd276-d95f-4c2a-8320-7b9302bd3769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:15.265561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccd276-d95f-4c2a-8320-7b9302bd3769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.265561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccd276-d95f-4c2a-8320-7b9302bd3769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.265561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccd276-d95f-4c2a-8320-7b9302bd3769 admin admin] PATH_INFO: `/v3/projects/1d6f575defed448684d597f86a4c25df` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.266432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deccd276-d95f-4c2a-8320-7b9302bd3769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d6f575defed448684d597f86a4c25df)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:15.268231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deccd276-d95f-4c2a-8320-7b9302bd3769 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:15.269146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4718210-8dcb-4b16-9681-6b8819c2b863 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:15.276501 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea774b07-7620-4ca2-aa95-d05312125dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:15.277870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea774b07-7620-4ca2-aa95-d05312125dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.279177 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea774b07-7620-4ca2-aa95-d05312125dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.279563 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea774b07-7620-4ca2-aa95-d05312125dc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.281056 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea774b07-7620-4ca2-aa95-d05312125dc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:15.281439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea774b07-7620-4ca2-aa95-d05312125dc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:15.283426 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea774b07-7620-4ca2-aa95-d05312125dc7 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:15.322010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 927/1887] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:15.325605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea774b07-7620-4ca2-aa95-d05312125dc7 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:15.343893 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea774b07-7620-4ca2-aa95-d05312125dc7 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:15.347350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-deccd276-d95f-4c2a-8320-7b9302bd3769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d6f575defed448684d597f86a4c25df', 'request_id': 'req-deccd276-d95f-4c2a-8320-7b9302bd3769'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.354983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-deccd276-d95f-4c2a-8320-7b9302bd3769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d6f575defed448684d597f86a4c25df', 'request_id': 'req-deccd276-d95f-4c2a-8320-7b9302bd3769'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.390488 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-deccd276-d95f-4c2a-8320-7b9302bd3769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d6f575defed448684d597f86a4c25df', 'request_id': 'req-deccd276-d95f-4c2a-8320-7b9302bd3769'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.408284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-deccd276-d95f-4c2a-8320-7b9302bd3769 admin admin] The token cache is being invalidate because project 1d6f575defed448684d597f86a4c25df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:15.409178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-deccd276-d95f-4c2a-8320-7b9302bd3769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.409987 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-deccd276-d95f-4c2a-8320-7b9302bd3769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.414494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 961/1888] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:15 2022] DELETE /identity/v3/projects/1d6f575defed448684d597f86a4c25df => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:15.415451 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccd276-d95f-4c2a-8320-7b9302bd3769 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:15.417033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-deccd276-d95f-4c2a-8320-7b9302bd3769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:15.429137 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc1042-1faf-4e04-9379-7d918bb9203e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:15.429759 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc1042-1faf-4e04-9379-7d918bb9203e admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.430040 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc1042-1faf-4e04-9379-7d918bb9203e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.430801 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc1042-1faf-4e04-9379-7d918bb9203e admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.431467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc1042-1faf-4e04-9379-7d918bb9203e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:15.433423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc1042-1faf-4e04-9379-7d918bb9203e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:15.462561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:15.475611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 928/1889] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:15 2022] POST /identity/v3/projects => generated 371 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:15.480660 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc1042-1faf-4e04-9379-7d918bb9203e admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:15.484145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-05fc1042-1faf-4e04-9379-7d918bb9203e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:15.546282 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccd276-d95f-4c2a-8320-7b9302bd3769 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:15.574978 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc1042-1faf-4e04-9379-7d918bb9203e admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:15.581462 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64d9fb-532f-463e-b1cd-532eb2890298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:15.582413 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64d9fb-532f-463e-b1cd-532eb2890298 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.582898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64d9fb-532f-463e-b1cd-532eb2890298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.583417 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64d9fb-532f-463e-b1cd-532eb2890298 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.584275 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64d9fb-532f-463e-b1cd-532eb2890298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:15.587132 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64d9fb-532f-463e-b1cd-532eb2890298 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:15.620733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 929/1890] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:15 2022] POST /identity/v3/users => generated 286 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:15.625907 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64d9fb-532f-463e-b1cd-532eb2890298 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:15.637382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64d9fb-532f-463e-b1cd-532eb2890298 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:15.638368 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ab515-acb1-4463-88f7-813cef39581f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:15.639101 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ab515-acb1-4463-88f7-813cef39581f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.639394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ab515-acb1-4463-88f7-813cef39581f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.639694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ab515-acb1-4463-88f7-813cef39581f admin admin] PATH_INFO: `/v3/users/7db724de032448f79e478a022581724a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.640939 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ab515-acb1-4463-88f7-813cef39581f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7db724de032448f79e478a022581724a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:15.642940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ab515-acb1-4463-88f7-813cef39581f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:15.643044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f237a-845e-47c1-94dd-cb132cbc8c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:15.643678 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f237a-845e-47c1-94dd-cb132cbc8c3e admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.643947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f237a-845e-47c1-94dd-cb132cbc8c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.644218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496f237a-845e-47c1-94dd-cb132cbc8c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.644865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f237a-845e-47c1-94dd-cb132cbc8c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:15.646448 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496f237a-845e-47c1-94dd-cb132cbc8c3e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:15.662836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 930/1891] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:15.663975 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f237a-845e-47c1-94dd-cb132cbc8c3e admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:15.676406 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496f237a-845e-47c1-94dd-cb132cbc8c3e admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:15.682346 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a89bc8-045c-494e-93ed-e784a8d7d8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:15.682999 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a89bc8-045c-494e-93ed-e784a8d7d8a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.683516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a89bc8-045c-494e-93ed-e784a8d7d8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.683826 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a89bc8-045c-494e-93ed-e784a8d7d8a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.684561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a89bc8-045c-494e-93ed-e784a8d7d8a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:15.686607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a89bc8-045c-494e-93ed-e784a8d7d8a9 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:15.698485 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 931/1892] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:15.700154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5d3ab515-acb1-4463-88f7-813cef39581f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7db724de032448f79e478a022581724a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.701651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5d3ab515-acb1-4463-88f7-813cef39581f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7db724de032448f79e478a022581724a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.703214 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a89bc8-045c-494e-93ed-e784a8d7d8a9 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:15.710364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5d3ab515-acb1-4463-88f7-813cef39581f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7db724de032448f79e478a022581724a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.720485 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a89bc8-045c-494e-93ed-e784a8d7d8a9 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:15.727696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5d3ab515-acb1-4463-88f7-813cef39581f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7db724de032448f79e478a022581724a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.730164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689316b5-eecc-49e4-aedd-cc05a6ef4246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:15.730844 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689316b5-eecc-49e4-aedd-cc05a6ef4246 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.733167 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689316b5-eecc-49e4-aedd-cc05a6ef4246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.733167 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689316b5-eecc-49e4-aedd-cc05a6ef4246 admin admin] PATH_INFO: `/v3/projects/22ca79a036874f679df3c5786415af46/users/95c7a58872224b0d867c8fdfc84f5212/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.759017 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:15.759233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5d3ab515-acb1-4463-88f7-813cef39581f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7db724de032448f79e478a022581724a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:15.768075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689316b5-eecc-49e4-aedd-cc05a6ef4246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22ca79a036874f679df3c5786415af46, user_id=95c7a58872224b0d867c8fdfc84f5212, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:15.774058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689316b5-eecc-49e4-aedd-cc05a6ef4246 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:15.777796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 962/1893] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:15 2022] DELETE /identity/v3/users/7db724de032448f79e478a022581724a => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:15.812750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 932/1894] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:15 2022] PUT /identity/v3/projects/22ca79a036874f679df3c5786415af46/users/95c7a58872224b0d867c8fdfc84f5212/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:15.817889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263169f3-2a30-4c8b-a532-a75fc4db5c32 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:15.818173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263169f3-2a30-4c8b-a532-a75fc4db5c32 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:15.818469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263169f3-2a30-4c8b-a532-a75fc4db5c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:15.861754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-263169f3-2a30-4c8b-a532-a75fc4db5c32 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:15.863163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-263169f3-2a30-4c8b-a532-a75fc4db5c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:15.930898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:15.970870 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 963/1895] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:15.993021 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689316b5-eecc-49e4-aedd-cc05a6ef4246 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:15.995254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-689316b5-eecc-49e4-aedd-cc05a6ef4246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:16.230407 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689316b5-eecc-49e4-aedd-cc05a6ef4246 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:16.258478 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d60867-833a-4bc7-b71b-9ffe9f3ec2cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:16.259559 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d60867-833a-4bc7-b71b-9ffe9f3ec2cf service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:16.260553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d60867-833a-4bc7-b71b-9ffe9f3ec2cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:16.261541 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d60867-833a-4bc7-b71b-9ffe9f3ec2cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:16.290017 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d60867-833a-4bc7-b71b-9ffe9f3ec2cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:16.292030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d60867-833a-4bc7-b71b-9ffe9f3ec2cf service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:16.303497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 933/1896] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:16.695552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263169f3-2a30-4c8b-a532-a75fc4db5c32 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:16.697070 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-263169f3-2a30-4c8b-a532-a75fc4db5c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:16.721976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263169f3-2a30-4c8b-a532-a75fc4db5c32 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:16.728179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f501589-b1c2-4a0a-bddb-e8ccbc8eb25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:16.728790 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f501589-b1c2-4a0a-bddb-e8ccbc8eb25e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:16.729618 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f501589-b1c2-4a0a-bddb-e8ccbc8eb25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:16.729618 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f501589-b1c2-4a0a-bddb-e8ccbc8eb25e admin admin] PATH_INFO: `/v3/projects/f513336a5d9141ce99f55c3b3bc08dd7` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:16.730747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f501589-b1c2-4a0a-bddb-e8ccbc8eb25e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f513336a5d9141ce99f55c3b3bc08dd7)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:16.732770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f501589-b1c2-4a0a-bddb-e8ccbc8eb25e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:16.796136 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0f501589-b1c2-4a0a-bddb-e8ccbc8eb25e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f513336a5d9141ce99f55c3b3bc08dd7', 'request_id': 'req-0f501589-b1c2-4a0a-bddb-e8ccbc8eb25e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:16.804019 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0f501589-b1c2-4a0a-bddb-e8ccbc8eb25e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f513336a5d9141ce99f55c3b3bc08dd7', 'request_id': 'req-0f501589-b1c2-4a0a-bddb-e8ccbc8eb25e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:16.814592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0f501589-b1c2-4a0a-bddb-e8ccbc8eb25e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f513336a5d9141ce99f55c3b3bc08dd7', 'request_id': 'req-0f501589-b1c2-4a0a-bddb-e8ccbc8eb25e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:16.827777 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0f501589-b1c2-4a0a-bddb-e8ccbc8eb25e admin admin] The token cache is being invalidate because project f513336a5d9141ce99f55c3b3bc08dd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:16.828074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0f501589-b1c2-4a0a-bddb-e8ccbc8eb25e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:16.828735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0f501589-b1c2-4a0a-bddb-e8ccbc8eb25e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:16.830472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 964/1897] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:16 2022] DELETE /identity/v3/projects/f513336a5d9141ce99f55c3b3bc08dd7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:16.837089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db2320-d588-4d55-b521-242dd3f89d92 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:16.837619 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db2320-d588-4d55-b521-242dd3f89d92 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:16.837736 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db2320-d588-4d55-b521-242dd3f89d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:16.864906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-09db2320-d588-4d55-b521-242dd3f89d92 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:16.866474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-09db2320-d588-4d55-b521-242dd3f89d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:16.960835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 934/1898] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:16.969091 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e7c3c-f016-4bcf-9dd4-1e790e47ca51 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:16.969614 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e7c3c-f016-4bcf-9dd4-1e790e47ca51 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:16.969799 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e7c3c-f016-4bcf-9dd4-1e790e47ca51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:16.995278 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-293e7c3c-f016-4bcf-9dd4-1e790e47ca51 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:16.996485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-293e7c3c-f016-4bcf-9dd4-1e790e47ca51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:17.007259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 965/1899] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:17.011265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db2320-d588-4d55-b521-242dd3f89d92 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:17.022223 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db2320-d588-4d55-b521-242dd3f89d92 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:17.027265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8354c-7e5f-466c-94b2-952fe7ed7f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:17.027949 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8354c-7e5f-466c-94b2-952fe7ed7f12 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.029769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8354c-7e5f-466c-94b2-952fe7ed7f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.030002 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8354c-7e5f-466c-94b2-952fe7ed7f12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.030790 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8354c-7e5f-466c-94b2-952fe7ed7f12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:17.031164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8354c-7e5f-466c-94b2-952fe7ed7f12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:17.032867 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8354c-7e5f-466c-94b2-952fe7ed7f12 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:17.046128 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 935/1900] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:17.050288 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e7c3c-f016-4bcf-9dd4-1e790e47ca51 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:17.060681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e7c3c-f016-4bcf-9dd4-1e790e47ca51 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:17.066662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dc55f-597d-44f5-a0ea-6876c5b7bf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:17.067353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dc55f-597d-44f5-a0ea-6876c5b7bf0c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.067658 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dc55f-597d-44f5-a0ea-6876c5b7bf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.067956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dc55f-597d-44f5-a0ea-6876c5b7bf0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.068590 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8dc55f-597d-44f5-a0ea-6876c5b7bf0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:17.070539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8dc55f-597d-44f5-a0ea-6876c5b7bf0c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:17.100208 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 966/1901] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:17 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:17.104621 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8354c-7e5f-466c-94b2-952fe7ed7f12 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:17.114881 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8354c-7e5f-466c-94b2-952fe7ed7f12 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:17.182300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48514840-6627-41ac-81e7-2f5b94816e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:17.183238 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48514840-6627-41ac-81e7-2f5b94816e2a admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.183787 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48514840-6627-41ac-81e7-2f5b94816e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.184288 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48514840-6627-41ac-81e7-2f5b94816e2a admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.185884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48514840-6627-41ac-81e7-2f5b94816e2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:17.187914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48514840-6627-41ac-81e7-2f5b94816e2a admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:17.217929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 936/1902] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:17 2022] POST /identity/v3/users => generated 284 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:17.219204 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dc55f-597d-44f5-a0ea-6876c5b7bf0c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:17.231367 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dc55f-597d-44f5-a0ea-6876c5b7bf0c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:17.238040 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c06be4-2192-49f5-983f-c9573b19243f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:17.239148 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c06be4-2192-49f5-983f-c9573b19243f admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.239527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c06be4-2192-49f5-983f-c9573b19243f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.239923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c06be4-2192-49f5-983f-c9573b19243f admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.240723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c06be4-2192-49f5-983f-c9573b19243f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:17.242622 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c06be4-2192-49f5-983f-c9573b19243f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:17.258789 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 967/1903] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:17.262687 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48514840-6627-41ac-81e7-2f5b94816e2a admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:17.275605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48514840-6627-41ac-81e7-2f5b94816e2a admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:17.282620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63405743-12f1-4b92-8e7b-27707c3eb013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:17.284164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63405743-12f1-4b92-8e7b-27707c3eb013 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.284454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63405743-12f1-4b92-8e7b-27707c3eb013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.286245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63405743-12f1-4b92-8e7b-27707c3eb013 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.287409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63405743-12f1-4b92-8e7b-27707c3eb013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:17.289571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63405743-12f1-4b92-8e7b-27707c3eb013 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:17.299662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:17.312334 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 937/1904] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:17.314530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c06be4-2192-49f5-983f-c9573b19243f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:17.337403 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871ee1e-df7c-489d-ac7a-c4c004ac9d1c None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.338160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871ee1e-df7c-489d-ac7a-c4c004ac9d1c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.338627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871ee1e-df7c-489d-ac7a-c4c004ac9d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.349599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c06be4-2192-49f5-983f-c9573b19243f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:17.356247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb9b84-353e-4ec7-a05e-cc626af2d3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:17.359119 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb9b84-353e-4ec7-a05e-cc626af2d3a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.359423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb9b84-353e-4ec7-a05e-cc626af2d3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.359785 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb9b84-353e-4ec7-a05e-cc626af2d3a7 admin admin] PATH_INFO: `/v3/projects/476e02daa1254c478297487abe5bbaf1/users/b3674d7b639f41179d4ff30bea8b522b/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.378500 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-5871ee1e-df7c-489d-ac7a-c4c004ac9d1c None None] MFA Rules not processed for user `95c7a58872224b0d867c8fdfc84f5212`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:17.378826 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb9b84-353e-4ec7-a05e-cc626af2d3a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=476e02daa1254c478297487abe5bbaf1, user_id=b3674d7b639f41179d4ff30bea8b522b, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:17.380064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-5871ee1e-df7c-489d-ac7a-c4c004ac9d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:17.381749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb9b84-353e-4ec7-a05e-cc626af2d3a7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:17.403924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 968/1905] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:17 2022] PUT /identity/v3/projects/476e02daa1254c478297487abe5bbaf1/users/b3674d7b639f41179d4ff30bea8b522b/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:17.411662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aece8a8-2490-4714-9b35-fc2edeab67b6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.412109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aece8a8-2490-4714-9b35-fc2edeab67b6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.412452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aece8a8-2490-4714-9b35-fc2edeab67b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.445258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2aece8a8-2490-4714-9b35-fc2edeab67b6 None None] MFA Rules not processed for user `b3674d7b639f41179d4ff30bea8b522b`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:17.446735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2aece8a8-2490-4714-9b35-fc2edeab67b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:17.470143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 938/1906] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:17.474870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871ee1e-df7c-489d-ac7a-c4c004ac9d1c None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:17.476693 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-5871ee1e-df7c-489d-ac7a-c4c004ac9d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:17.492721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871ee1e-df7c-489d-ac7a-c4c004ac9d1c None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:17.501893 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:17.524109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 969/1907] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:17.536177 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aece8a8-2490-4714-9b35-fc2edeab67b6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:17.537591 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2aece8a8-2490-4714-9b35-fc2edeab67b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:17.573930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c12e0-48ac-48f9-839c-1f507b5fbc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:17.574720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c12e0-48ac-48f9-839c-1f507b5fbc8f admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.575066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c12e0-48ac-48f9-839c-1f507b5fbc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.575489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c12e0-48ac-48f9-839c-1f507b5fbc8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.576514 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c12e0-48ac-48f9-839c-1f507b5fbc8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:17.579002 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c12e0-48ac-48f9-839c-1f507b5fbc8f admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:17.623382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 939/1908] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:17 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:17.626819 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c12e0-48ac-48f9-839c-1f507b5fbc8f admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:17.654768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aece8a8-2490-4714-9b35-fc2edeab67b6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:17.711400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c12e0-48ac-48f9-839c-1f507b5fbc8f admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:17.712524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4dc6a-c661-4edc-824a-2988c081919c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd8df2f3357d4660bc22366f9746f89c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:17.713231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4dc6a-c661-4edc-824a-2988c081919c service glance] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.713514 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4dc6a-c661-4edc-824a-2988c081919c service glance] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.713789 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4dc6a-c661-4edc-824a-2988c081919c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.716844 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d30f8-343a-49cb-b1f0-39c245966674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:17.717461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d30f8-343a-49cb-b1f0-39c245966674 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.717740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d30f8-343a-49cb-b1f0-39c245966674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.718670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d30f8-343a-49cb-b1f0-39c245966674 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.718670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d30f8-343a-49cb-b1f0-39c245966674 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:17.720696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d30f8-343a-49cb-b1f0-39c245966674 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:17.743471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 940/1909] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:17 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:17.747026 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d30f8-343a-49cb-b1f0-39c245966674 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:17.754301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:17.758158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d30f8-343a-49cb-b1f0-39c245966674 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:17.766269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6934ac-052e-46ea-9dc1-63e92dc77640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:17.766884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6934ac-052e-46ea-9dc1-63e92dc77640 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.767169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6934ac-052e-46ea-9dc1-63e92dc77640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.767443 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6934ac-052e-46ea-9dc1-63e92dc77640 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.768199 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6934ac-052e-46ea-9dc1-63e92dc77640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:17.769805 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6934ac-052e-46ea-9dc1-63e92dc77640 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:17.782703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 941/1910] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:17.786942 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6934ac-052e-46ea-9dc1-63e92dc77640 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:17.789746 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4dc6a-c661-4edc-824a-2988c081919c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:17.791214 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa4dc6a-c661-4edc-824a-2988c081919c service glance] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:17.798824 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6934ac-052e-46ea-9dc1-63e92dc77640 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:17.801698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 970/1911] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:17.805551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d6ba3-90c7-4980-b1de-813510318c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:17.806454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d6ba3-90c7-4980-b1de-813510318c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.807007 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d6ba3-90c7-4980-b1de-813510318c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.807488 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d6ba3-90c7-4980-b1de-813510318c8f admin admin] PATH_INFO: `/v3/projects/8cbee61459cb40b1b8453beb77eac4c8/users/bb22418d4e6343f7bc2857c36be1e6a8/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.824584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d6ba3-90c7-4980-b1de-813510318c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbee61459cb40b1b8453beb77eac4c8, user_id=bb22418d4e6343f7bc2857c36be1e6a8, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:17.828279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d6ba3-90c7-4980-b1de-813510318c8f admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:17.860301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 942/1912] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:17 2022] PUT /identity/v3/projects/8cbee61459cb40b1b8453beb77eac4c8/users/bb22418d4e6343f7bc2857c36be1e6a8/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:17.864281 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4dc6a-c661-4edc-824a-2988c081919c service glance] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:17.879038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4dc6a-c661-4edc-824a-2988c081919c service glance] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:17.925893 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:17.957935 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c22d3-79c9-4c7a-bfb8-22b5a1682c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:17.958631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c22d3-79c9-4c7a-bfb8-22b5a1682c5c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:17.958987 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c22d3-79c9-4c7a-bfb8-22b5a1682c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:17.959335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c22d3-79c9-4c7a-bfb8-22b5a1682c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:17.960389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c22d3-79c9-4c7a-bfb8-22b5a1682c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:17.962437 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c22d3-79c9-4c7a-bfb8-22b5a1682c5c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:17.979838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 971/1913] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:17 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:17.981401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d6ba3-90c7-4980-b1de-813510318c8f admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:17.993997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d6ba3-90c7-4980-b1de-813510318c8f admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:17.999527 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca63c54-4aae-4f38-aa45-89c6da7af73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:18.000282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca63c54-4aae-4f38-aa45-89c6da7af73c admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:18.000578 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca63c54-4aae-4f38-aa45-89c6da7af73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:18.000858 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca63c54-4aae-4f38-aa45-89c6da7af73c admin admin] PATH_INFO: `/v3/projects/8cbee61459cb40b1b8453beb77eac4c8/users/bb22418d4e6343f7bc2857c36be1e6a8/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:18.003062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca63c54-4aae-4f38-aa45-89c6da7af73c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cbee61459cb40b1b8453beb77eac4c8, user_id=bb22418d4e6343f7bc2857c36be1e6a8, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:18.005891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca63c54-4aae-4f38-aa45-89c6da7af73c admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:18.017902 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 943/1914] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:17 2022] PUT /identity/v3/projects/8cbee61459cb40b1b8453beb77eac4c8/users/bb22418d4e6343f7bc2857c36be1e6a8/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:19.024142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ac0cc-6bd4-4f48-8218-a4ff1327c2f0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:19.024646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ac0cc-6bd4-4f48-8218-a4ff1327c2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:19.024738 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ac0cc-6bd4-4f48-8218-a4ff1327c2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:19.057941 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-cd8ac0cc-6bd4-4f48-8218-a4ff1327c2f0 None None] MFA Rules not processed for user `bb22418d4e6343f7bc2857c36be1e6a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:19.060299 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cd8ac0cc-6bd4-4f48-8218-a4ff1327c2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:19.229361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 972/1915] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:19.246986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca63c54-4aae-4f38-aa45-89c6da7af73c admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:19.249431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2ca63c54-4aae-4f38-aa45-89c6da7af73c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:19.379914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca63c54-4aae-4f38-aa45-89c6da7af73c admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:19.401198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389735b1-accd-4311-a473-693fd9000d05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:19.402570 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389735b1-accd-4311-a473-693fd9000d05 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:19.402911 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389735b1-accd-4311-a473-693fd9000d05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:19.403236 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389735b1-accd-4311-a473-693fd9000d05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:19.610551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389735b1-accd-4311-a473-693fd9000d05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:19.614172 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389735b1-accd-4311-a473-693fd9000d05 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:19.667947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 944/1916] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:21.209447 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ac0cc-6bd4-4f48-8218-a4ff1327c2f0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:21.211528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cd8ac0cc-6bd4-4f48-8218-a4ff1327c2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:21.226962 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ac0cc-6bd4-4f48-8218-a4ff1327c2f0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:21.237404 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd902248-2a1b-40e6-b8a7-9392ebfd4534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:21.238374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd902248-2a1b-40e6-b8a7-9392ebfd4534 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:21.238834 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd902248-2a1b-40e6-b8a7-9392ebfd4534 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:21.239292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd902248-2a1b-40e6-b8a7-9392ebfd4534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:21.251909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd902248-2a1b-40e6-b8a7-9392ebfd4534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:21.253521 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd902248-2a1b-40e6-b8a7-9392ebfd4534 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:21.267808 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 973/1917] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:23.272823 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389735b1-accd-4311-a473-693fd9000d05 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:23.274294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-389735b1-accd-4311-a473-693fd9000d05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:23.400843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389735b1-accd-4311-a473-693fd9000d05 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:23.405902 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b09b8f-9de4-4b97-9395-a78a3d23390c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:23.406578 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b09b8f-9de4-4b97-9395-a78a3d23390c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:23.406866 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b09b8f-9de4-4b97-9395-a78a3d23390c admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:23.407909 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b09b8f-9de4-4b97-9395-a78a3d23390c admin admin] PATH_INFO: `/v3/users/4ed8274584ca409bafc99b3e6e6a1ec1` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:23.408590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b09b8f-9de4-4b97-9395-a78a3d23390c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ed8274584ca409bafc99b3e6e6a1ec1)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:23.410669 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b09b8f-9de4-4b97-9395-a78a3d23390c admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:23.425679 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:23.475951 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-99b09b8f-9de4-4b97-9395-a78a3d23390c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ed8274584ca409bafc99b3e6e6a1ec1'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:23.476642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-99b09b8f-9de4-4b97-9395-a78a3d23390c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ed8274584ca409bafc99b3e6e6a1ec1'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:23.477240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-99b09b8f-9de4-4b97-9395-a78a3d23390c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ed8274584ca409bafc99b3e6e6a1ec1'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:23.487770 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-99b09b8f-9de4-4b97-9395-a78a3d23390c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ed8274584ca409bafc99b3e6e6a1ec1'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:23.498020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-99b09b8f-9de4-4b97-9395-a78a3d23390c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ed8274584ca409bafc99b3e6e6a1ec1'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:23.520148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 945/1918] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:23 2022] DELETE /identity/v3/users/4ed8274584ca409bafc99b3e6e6a1ec1 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:23.524715 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82189a-85ec-409c-9fa8-363a09c3d4b1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:23.525197 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82189a-85ec-409c-9fa8-363a09c3d4b1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:23.525274 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e82189a-85ec-409c-9fa8-363a09c3d4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:23.545048 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4e82189a-85ec-409c-9fa8-363a09c3d4b1 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:23.547170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4e82189a-85ec-409c-9fa8-363a09c3d4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:23.633323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 974/1919] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:23.644289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b09b8f-9de4-4b97-9395-a78a3d23390c admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:23.646701 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-99b09b8f-9de4-4b97-9395-a78a3d23390c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:23.727332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b09b8f-9de4-4b97-9395-a78a3d23390c admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:23.731471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892b524-95e2-4ca5-b4c5-da4ac518d756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:23.732206 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892b524-95e2-4ca5-b4c5-da4ac518d756 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:23.732610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892b524-95e2-4ca5-b4c5-da4ac518d756 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:23.735783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892b524-95e2-4ca5-b4c5-da4ac518d756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:23.747388 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892b524-95e2-4ca5-b4c5-da4ac518d756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:23.749074 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892b524-95e2-4ca5-b4c5-da4ac518d756 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:23.773781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 946/1920] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:24.224552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82189a-85ec-409c-9fa8-363a09c3d4b1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:24.226850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4e82189a-85ec-409c-9fa8-363a09c3d4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:24.248253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e82189a-85ec-409c-9fa8-363a09c3d4b1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:24.256328 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878adea6-9599-4944-ab80-6ceeb72ecedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:24.257263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878adea6-9599-4944-ab80-6ceeb72ecedd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:24.257732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878adea6-9599-4944-ab80-6ceeb72ecedd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:24.258190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878adea6-9599-4944-ab80-6ceeb72ecedd admin admin] PATH_INFO: `/v3/projects/5b91c5b04b42452789555241d7ae1faf` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:24.259547 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878adea6-9599-4944-ab80-6ceeb72ecedd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b91c5b04b42452789555241d7ae1faf)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:24.262333 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878adea6-9599-4944-ab80-6ceeb72ecedd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:24.310384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:24.315118 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-878adea6-9599-4944-ab80-6ceeb72ecedd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b91c5b04b42452789555241d7ae1faf', 'request_id': 'req-878adea6-9599-4944-ab80-6ceeb72ecedd'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.321235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-878adea6-9599-4944-ab80-6ceeb72ecedd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b91c5b04b42452789555241d7ae1faf', 'request_id': 'req-878adea6-9599-4944-ab80-6ceeb72ecedd'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.330088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-878adea6-9599-4944-ab80-6ceeb72ecedd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b91c5b04b42452789555241d7ae1faf', 'request_id': 'req-878adea6-9599-4944-ab80-6ceeb72ecedd'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.341834 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-878adea6-9599-4944-ab80-6ceeb72ecedd admin admin] The token cache is being invalidate because project 5b91c5b04b42452789555241d7ae1faf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:24.342204 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-878adea6-9599-4944-ab80-6ceeb72ecedd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.342963 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-878adea6-9599-4944-ab80-6ceeb72ecedd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.345423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 975/1921] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:24 2022] DELETE /identity/v3/projects/5b91c5b04b42452789555241d7ae1faf => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:24.348654 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892b524-95e2-4ca5-b4c5-da4ac518d756 service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:24.359111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892b524-95e2-4ca5-b4c5-da4ac518d756 service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:24.449772 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0c8a88-cefe-4153-89b6-ac62df9f11c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:24.450544 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c8a88-cefe-4153-89b6-ac62df9f11c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:24.450829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c8a88-cefe-4153-89b6-ac62df9f11c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:24.451109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0c8a88-cefe-4153-89b6-ac62df9f11c4 admin admin] PATH_INFO: `/v3/users/f2456a6677fa4408a91fd4c3a2488ebf` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:24.453091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c8a88-cefe-4153-89b6-ac62df9f11c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2456a6677fa4408a91fd4c3a2488ebf)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:24.455095 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0c8a88-cefe-4153-89b6-ac62df9f11c4 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:24.532335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:24.566070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-df0c8a88-cefe-4153-89b6-ac62df9f11c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2456a6677fa4408a91fd4c3a2488ebf'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.566993 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-df0c8a88-cefe-4153-89b6-ac62df9f11c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2456a6677fa4408a91fd4c3a2488ebf'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.568377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-df0c8a88-cefe-4153-89b6-ac62df9f11c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2456a6677fa4408a91fd4c3a2488ebf'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.583112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-df0c8a88-cefe-4153-89b6-ac62df9f11c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2456a6677fa4408a91fd4c3a2488ebf'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.592493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-df0c8a88-cefe-4153-89b6-ac62df9f11c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2456a6677fa4408a91fd4c3a2488ebf'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:24.618133 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 947/1922] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:24 2022] DELETE /identity/v3/users/f2456a6677fa4408a91fd4c3a2488ebf => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:25.356929 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878adea6-9599-4944-ab80-6ceeb72ecedd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:25.358504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-878adea6-9599-4944-ab80-6ceeb72ecedd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:25.380304 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878adea6-9599-4944-ab80-6ceeb72ecedd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:25.494569 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d530f9-0b8e-4247-8516-f210daf1b41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:25.496161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d530f9-0b8e-4247-8516-f210daf1b41b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:25.496665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d530f9-0b8e-4247-8516-f210daf1b41b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:25.497171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d530f9-0b8e-4247-8516-f210daf1b41b admin admin] PATH_INFO: `/v3/projects/c41697d4a34f4c7fb6e8bbc4b4cf5933` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:25.498611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d530f9-0b8e-4247-8516-f210daf1b41b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c41697d4a34f4c7fb6e8bbc4b4cf5933)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:25.502181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d530f9-0b8e-4247-8516-f210daf1b41b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:25.587063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d7d530f9-0b8e-4247-8516-f210daf1b41b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c41697d4a34f4c7fb6e8bbc4b4cf5933', 'request_id': 'req-d7d530f9-0b8e-4247-8516-f210daf1b41b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.598766 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d7d530f9-0b8e-4247-8516-f210daf1b41b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c41697d4a34f4c7fb6e8bbc4b4cf5933', 'request_id': 'req-d7d530f9-0b8e-4247-8516-f210daf1b41b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.611996 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d7d530f9-0b8e-4247-8516-f210daf1b41b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c41697d4a34f4c7fb6e8bbc4b4cf5933', 'request_id': 'req-d7d530f9-0b8e-4247-8516-f210daf1b41b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.636369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d7d530f9-0b8e-4247-8516-f210daf1b41b admin admin] The token cache is being invalidate because project c41697d4a34f4c7fb6e8bbc4b4cf5933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:25.636745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d7d530f9-0b8e-4247-8516-f210daf1b41b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.639520 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d7d530f9-0b8e-4247-8516-f210daf1b41b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:25.645165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 976/1923] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:25 2022] DELETE /identity/v3/projects/c41697d4a34f4c7fb6e8bbc4b4cf5933 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:25.646791 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcd0a4-f927-48b7-a7fe-fc3e003d1bec None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:25.647261 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcd0a4-f927-48b7-a7fe-fc3e003d1bec None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:25.647366 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fcd0a4-f927-48b7-a7fe-fc3e003d1bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:25.691867 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-a4fcd0a4-f927-48b7-a7fe-fc3e003d1bec None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:25.693612 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a4fcd0a4-f927-48b7-a7fe-fc3e003d1bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:25.814092 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0096c-4ab7-411e-8c1c-a6716bd6aba6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:25.814877 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0096c-4ab7-411e-8c1c-a6716bd6aba6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:25.815052 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 948/1924] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:25.815324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0096c-4ab7-411e-8c1c-a6716bd6aba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:25.853224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3ae0096c-4ab7-411e-8c1c-a6716bd6aba6 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:25.882120 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3ae0096c-4ab7-411e-8c1c-a6716bd6aba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:25.913570 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcd0a4-f927-48b7-a7fe-fc3e003d1bec None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:25.927554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fcd0a4-f927-48b7-a7fe-fc3e003d1bec None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:25.928216 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 977/1925] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:25.943792 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6513d-60ff-4299-a836-6a7d941e44e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:25.944436 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6513d-60ff-4299-a836-6a7d941e44e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:25.944755 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6513d-60ff-4299-a836-6a7d941e44e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:25.944983 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6513d-60ff-4299-a836-6a7d941e44e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:25.945722 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6513d-60ff-4299-a836-6a7d941e44e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:25.945951 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6513d-60ff-4299-a836-6a7d941e44e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:25.947587 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6513d-60ff-4299-a836-6a7d941e44e0 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:25.991576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0096c-4ab7-411e-8c1c-a6716bd6aba6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.004015 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0096c-4ab7-411e-8c1c-a6716bd6aba6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.009530 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 949/1926] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:33:26.024213 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e79fd8-1612-4757-84da-ab8193e33edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.025382 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e79fd8-1612-4757-84da-ab8193e33edf admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.025815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e79fd8-1612-4757-84da-ab8193e33edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.026100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e79fd8-1612-4757-84da-ab8193e33edf admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.026738 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e79fd8-1612-4757-84da-ab8193e33edf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.028723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e79fd8-1612-4757-84da-ab8193e33edf admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.096078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 978/1927] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:25 2022] POST /identity/v3/projects => generated 355 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:26.100418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6513d-60ff-4299-a836-6a7d941e44e0 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.116942 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6513d-60ff-4299-a836-6a7d941e44e0 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.243720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb00c67-55e2-4a0f-a2ba-2c8d6348d2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.244529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb00c67-55e2-4a0f-a2ba-2c8d6348d2d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.244732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb00c67-55e2-4a0f-a2ba-2c8d6348d2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.245022 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb00c67-55e2-4a0f-a2ba-2c8d6348d2d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.245892 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb00c67-55e2-4a0f-a2ba-2c8d6348d2d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.248419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb00c67-55e2-4a0f-a2ba-2c8d6348d2d4 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.273772 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 950/1928] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:26 2022] POST /identity/v3/users => generated 278 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:26.277745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e79fd8-1612-4757-84da-ab8193e33edf admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.288979 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e79fd8-1612-4757-84da-ab8193e33edf admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.294507 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b229d-be5d-4ddb-9abe-0a168c9f3d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.295135 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b229d-be5d-4ddb-9abe-0a168c9f3d4b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.295420 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b229d-be5d-4ddb-9abe-0a168c9f3d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.300620 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b229d-be5d-4ddb-9abe-0a168c9f3d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.301387 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b229d-be5d-4ddb-9abe-0a168c9f3d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.303082 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b229d-be5d-4ddb-9abe-0a168c9f3d4b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.319778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 979/1929] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:26.320852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb00c67-55e2-4a0f-a2ba-2c8d6348d2d4 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.331641 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb00c67-55e2-4a0f-a2ba-2c8d6348d2d4 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.336244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0f7e75-015e-40fc-95f8-73129ca4bb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.336847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f7e75-015e-40fc-95f8-73129ca4bb2f admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.337126 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f7e75-015e-40fc-95f8-73129ca4bb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.337400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0f7e75-015e-40fc-95f8-73129ca4bb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.338042 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f7e75-015e-40fc-95f8-73129ca4bb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.339648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0f7e75-015e-40fc-95f8-73129ca4bb2f admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.351003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 951/1930] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:26.355381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b229d-be5d-4ddb-9abe-0a168c9f3d4b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.366220 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b229d-be5d-4ddb-9abe-0a168c9f3d4b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.371512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9621d0a-0295-4c2f-9ce7-6456c21d3b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.372203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9621d0a-0295-4c2f-9ce7-6456c21d3b1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.372490 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9621d0a-0295-4c2f-9ce7-6456c21d3b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.372770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9621d0a-0295-4c2f-9ce7-6456c21d3b1d admin admin] PATH_INFO: `/v3/projects/1360541ac75e4cb6b58b607698fc4711/users/39bb5639c8284c69809cc9f206bbbfb7/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.388667 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9621d0a-0295-4c2f-9ce7-6456c21d3b1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1360541ac75e4cb6b58b607698fc4711, user_id=39bb5639c8284c69809cc9f206bbbfb7, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.391478 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9621d0a-0295-4c2f-9ce7-6456c21d3b1d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.400499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:26.410624 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 980/1931] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:26 2022] PUT /identity/v3/projects/1360541ac75e4cb6b58b607698fc4711/users/39bb5639c8284c69809cc9f206bbbfb7/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:26.420488 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d2968-53d1-4d15-98a1-192569109f09 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.422577 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d2968-53d1-4d15-98a1-192569109f09 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.422873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d2968-53d1-4d15-98a1-192569109f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.453106 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-ca9d2968-53d1-4d15-98a1-192569109f09 None None] MFA Rules not processed for user `39bb5639c8284c69809cc9f206bbbfb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:26.454497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-ca9d2968-53d1-4d15-98a1-192569109f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:26.460523 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:26.527323 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 952/1932] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:26.532620 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9621d0a-0295-4c2f-9ce7-6456c21d3b1d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.622886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9621d0a-0295-4c2f-9ce7-6456c21d3b1d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.641397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887fc29-7d3b-4234-af29-30f9f515e208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.642038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887fc29-7d3b-4234-af29-30f9f515e208 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.642323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887fc29-7d3b-4234-af29-30f9f515e208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.642599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887fc29-7d3b-4234-af29-30f9f515e208 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.643240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887fc29-7d3b-4234-af29-30f9f515e208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.645205 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887fc29-7d3b-4234-af29-30f9f515e208 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.674001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 981/1933] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:26 2022] POST /identity/v3/projects => generated 355 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:26.678032 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d2968-53d1-4d15-98a1-192569109f09 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.689126 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d2968-53d1-4d15-98a1-192569109f09 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.793144 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f695761-536b-4d54-bbe8-76ea7a24c940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.793767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f695761-536b-4d54-bbe8-76ea7a24c940 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.794049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f695761-536b-4d54-bbe8-76ea7a24c940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.794330 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f695761-536b-4d54-bbe8-76ea7a24c940 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.794966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f695761-536b-4d54-bbe8-76ea7a24c940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.797555 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f695761-536b-4d54-bbe8-76ea7a24c940 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.819685 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 953/1934] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:26 2022] POST /identity/v3/users => generated 284 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:26.823749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887fc29-7d3b-4234-af29-30f9f515e208 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.836253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887fc29-7d3b-4234-af29-30f9f515e208 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.845081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353952d-2548-41f5-abb3-3db58113b366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.846825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353952d-2548-41f5-abb3-3db58113b366 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.846825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353952d-2548-41f5-abb3-3db58113b366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.846825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353952d-2548-41f5-abb3-3db58113b366 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.849252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353952d-2548-41f5-abb3-3db58113b366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.850161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3353952d-2548-41f5-abb3-3db58113b366 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.870226 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 982/1935] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:26.872262 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f695761-536b-4d54-bbe8-76ea7a24c940 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.891364 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f695761-536b-4d54-bbe8-76ea7a24c940 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:26.896914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55b8b0-571c-4cc3-acfd-fa329eebd0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:26.897599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55b8b0-571c-4cc3-acfd-fa329eebd0eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:26.897879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55b8b0-571c-4cc3-acfd-fa329eebd0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:26.898335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55b8b0-571c-4cc3-acfd-fa329eebd0eb admin admin] PATH_INFO: `/v3/projects/baf6370948eb4bf7a437d8127b23c0ae/users/7ffa4f761096401881b2765b266ef432/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:26.915605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55b8b0-571c-4cc3-acfd-fa329eebd0eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=baf6370948eb4bf7a437d8127b23c0ae, user_id=7ffa4f761096401881b2765b266ef432, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:26.918289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b55b8b0-571c-4cc3-acfd-fa329eebd0eb admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:26.931777 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 954/1936] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:26 2022] PUT /identity/v3/projects/baf6370948eb4bf7a437d8127b23c0ae/users/7ffa4f761096401881b2765b266ef432/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:26.935809 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353952d-2548-41f5-abb3-3db58113b366 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:26.946687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353952d-2548-41f5-abb3-3db58113b366 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.011426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d3191-686c-47ad-9850-be1a6ba89349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.012131 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d3191-686c-47ad-9850-be1a6ba89349 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.012433 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d3191-686c-47ad-9850-be1a6ba89349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.012695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d3191-686c-47ad-9850-be1a6ba89349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.013394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d3191-686c-47ad-9850-be1a6ba89349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.016315 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d3191-686c-47ad-9850-be1a6ba89349 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.027116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 983/1937] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:27.031477 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55b8b0-571c-4cc3-acfd-fa329eebd0eb admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.042263 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b55b8b0-571c-4cc3-acfd-fa329eebd0eb admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.046925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95016e64-6026-4a2b-8e18-7159a3ad4d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.047593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95016e64-6026-4a2b-8e18-7159a3ad4d45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.047899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95016e64-6026-4a2b-8e18-7159a3ad4d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.048174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95016e64-6026-4a2b-8e18-7159a3ad4d45 admin admin] PATH_INFO: `/v3/projects/baf6370948eb4bf7a437d8127b23c0ae/users/7ffa4f761096401881b2765b266ef432/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.051019 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95016e64-6026-4a2b-8e18-7159a3ad4d45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baf6370948eb4bf7a437d8127b23c0ae, user_id=7ffa4f761096401881b2765b266ef432, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.053709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95016e64-6026-4a2b-8e18-7159a3ad4d45 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.068171 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 955/1938] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:27 2022] PUT /identity/v3/projects/baf6370948eb4bf7a437d8127b23c0ae/users/7ffa4f761096401881b2765b266ef432/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:27.080104 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90de24-8628-420c-9872-abcb487e8843 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.080275 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90de24-8628-420c-9872-abcb487e8843 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.080602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90de24-8628-420c-9872-abcb487e8843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.123645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-4d90de24-8628-420c-9872-abcb487e8843 None None] MFA Rules not processed for user `7ffa4f761096401881b2765b266ef432`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:27.125984 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4d90de24-8628-420c-9872-abcb487e8843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:27.230614 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:27.252691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 984/1939] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:27.260487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95016e64-6026-4a2b-8e18-7159a3ad4d45 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.262831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-95016e64-6026-4a2b-8e18-7159a3ad4d45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:27.440692 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95016e64-6026-4a2b-8e18-7159a3ad4d45 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.444690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b9661-160d-4efd-8744-b15d5f37d630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.445498 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b9661-160d-4efd-8744-b15d5f37d630 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.445784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b9661-160d-4efd-8744-b15d5f37d630 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.446060 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b9661-160d-4efd-8744-b15d5f37d630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.457662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b9661-160d-4efd-8744-b15d5f37d630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.459288 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b9661-160d-4efd-8744-b15d5f37d630 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.481270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 956/1940] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:27 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:27.593444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90de24-8628-420c-9872-abcb487e8843 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:27.610995 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90de24-8628-420c-9872-abcb487e8843 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:27.615511 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e0f0d-cb54-4aee-b399-e08e70e5d0a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:27.616336 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e0f0d-cb54-4aee-b399-e08e70e5d0a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:27.616653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e0f0d-cb54-4aee-b399-e08e70e5d0a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:27.617004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e0f0d-cb54-4aee-b399-e08e70e5d0a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:27.724310 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e0f0d-cb54-4aee-b399-e08e70e5d0a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:27.726058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e0f0d-cb54-4aee-b399-e08e70e5d0a8 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:27.738063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 985/1941] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:27 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:35.910648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c8f22-03f1-4b47-a980-4ff46679a7a4 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:35.911158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c8f22-03f1-4b47-a980-4ff46679a7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:35.911238 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c8f22-03f1-4b47-a980-4ff46679a7a4 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:35.912627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 957/1942] 10.222.0.58 () {54 vars in 921 bytes} [Wed Aug 24 12:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:33:35.923557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753f530-01a7-4abb-87b1-6a127848035c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:35.924085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753f530-01a7-4abb-87b1-6a127848035c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:35.924231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753f530-01a7-4abb-87b1-6a127848035c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:35.944108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c753f530-01a7-4abb-87b1-6a127848035c None None] MFA Rules not processed for user `b7ba45e3cff74962965b248f9f017f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:35.945311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c753f530-01a7-4abb-87b1-6a127848035c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:36.026548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 986/1943] 10.222.0.58 () {58 vars in 1018 bytes} [Wed Aug 24 12:33:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:36.039162 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c8f22-03f1-4b47-a980-4ff46679a7a4 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:36.041022 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1c5c8f22-03f1-4b47-a980-4ff46679a7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:36.052075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:36.060182 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c8f22-03f1-4b47-a980-4ff46679a7a4 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:36.066519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e605d8d-adbd-460f-a9ba-16f45a9d8ad2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ba45e3cff74962965b248f9f017f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:36.067138 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e605d8d-adbd-460f-a9ba-16f45a9d8ad2 service placement] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:36.067418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e605d8d-adbd-460f-a9ba-16f45a9d8ad2 service placement] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:36.067726 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e605d8d-adbd-460f-a9ba-16f45a9d8ad2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:36.076554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e605d8d-adbd-460f-a9ba-16f45a9d8ad2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:36.078985 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e605d8d-adbd-460f-a9ba-16f45a9d8ad2 service placement] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:36.091075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 958/1944] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:36.528999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ee869-0b5d-4ce0-a556-ff7d034a7d03 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:36.529252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ee869-0b5d-4ce0-a556-ff7d034a7d03 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:36.529497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ee869-0b5d-4ce0-a556-ff7d034a7d03 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:36.531896 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 987/1945] 10.222.0.58 () {54 vars in 921 bytes} [Wed Aug 24 12:33:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:33:36.541969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6f80b-ac0c-4dea-aea1-1d12c1e8aedb None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:36.542531 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6f80b-ac0c-4dea-aea1-1d12c1e8aedb None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:36.542986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6f80b-ac0c-4dea-aea1-1d12c1e8aedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:36.564660 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-33b6f80b-ac0c-4dea-aea1-1d12c1e8aedb None None] MFA Rules not processed for user `b7ba45e3cff74962965b248f9f017f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:36.565886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-33b6f80b-ac0c-4dea-aea1-1d12c1e8aedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:36.575483 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 959/1946] 10.222.0.58 () {58 vars in 1018 bytes} [Wed Aug 24 12:33:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:36.598594 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ee869-0b5d-4ce0-a556-ff7d034a7d03 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:36.621288 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ee869-0b5d-4ce0-a556-ff7d034a7d03 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:36.626131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6f80b-ac0c-4dea-aea1-1d12c1e8aedb None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:36.627571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-33b6f80b-ac0c-4dea-aea1-1d12c1e8aedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:36.631205 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca6b19-32c4-45fb-9a24-33cfb048c7d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ba45e3cff74962965b248f9f017f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:36.632866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca6b19-32c4-45fb-9a24-33cfb048c7d5 service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:36.633364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca6b19-32c4-45fb-9a24-33cfb048c7d5 service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:36.633649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca6b19-32c4-45fb-9a24-33cfb048c7d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:36.644323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca6b19-32c4-45fb-9a24-33cfb048c7d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:36.646129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca6b19-32c4-45fb-9a24-33cfb048c7d5 service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:36.656877 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 988/1947] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:36.725354 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6f80b-ac0c-4dea-aea1-1d12c1e8aedb None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:36.730449 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4027af-9a9b-4707-a4e0-3999f3fc71f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:36.730669 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4027af-9a9b-4707-a4e0-3999f3fc71f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:36.731103 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4027af-9a9b-4707-a4e0-3999f3fc71f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:36.731306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4027af-9a9b-4707-a4e0-3999f3fc71f9 admin admin] PATH_INFO: `/v3/users/95c7a58872224b0d867c8fdfc84f5212` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:36.732362 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4027af-9a9b-4707-a4e0-3999f3fc71f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c7a58872224b0d867c8fdfc84f5212)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:36.735189 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4027af-9a9b-4707-a4e0-3999f3fc71f9 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:36.789258 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7a4027af-9a9b-4707-a4e0-3999f3fc71f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c7a58872224b0d867c8fdfc84f5212'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:36.789833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7a4027af-9a9b-4707-a4e0-3999f3fc71f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c7a58872224b0d867c8fdfc84f5212'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:36.790618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7a4027af-9a9b-4707-a4e0-3999f3fc71f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c7a58872224b0d867c8fdfc84f5212'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:36.798790 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7a4027af-9a9b-4707-a4e0-3999f3fc71f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c7a58872224b0d867c8fdfc84f5212'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:36.807052 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7a4027af-9a9b-4707-a4e0-3999f3fc71f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c7a58872224b0d867c8fdfc84f5212'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:36.828184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 960/1948] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:36 2022] DELETE /identity/v3/users/95c7a58872224b0d867c8fdfc84f5212 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:37.038476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca6b19-32c4-45fb-9a24-33cfb048c7d5 service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:37.064501 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca6b19-32c4-45fb-9a24-33cfb048c7d5 service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:37.135191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27dd11-7d11-4fdc-96ff-c744ad54004c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:37.135825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27dd11-7d11-4fdc-96ff-c744ad54004c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:37.136215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27dd11-7d11-4fdc-96ff-c744ad54004c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:37.136516 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27dd11-7d11-4fdc-96ff-c744ad54004c admin admin] PATH_INFO: `/v3/projects/22ca79a036874f679df3c5786415af46` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:37.137787 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27dd11-7d11-4fdc-96ff-c744ad54004c admin admin] RBAC: Authorizing `identity:delete_project(project_id=22ca79a036874f679df3c5786415af46)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:37.139868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27dd11-7d11-4fdc-96ff-c744ad54004c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:37.158297 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:37.194022 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8b27dd11-7d11-4fdc-96ff-c744ad54004c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22ca79a036874f679df3c5786415af46', 'request_id': 'req-8b27dd11-7d11-4fdc-96ff-c744ad54004c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.199971 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8b27dd11-7d11-4fdc-96ff-c744ad54004c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22ca79a036874f679df3c5786415af46', 'request_id': 'req-8b27dd11-7d11-4fdc-96ff-c744ad54004c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.208455 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8b27dd11-7d11-4fdc-96ff-c744ad54004c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22ca79a036874f679df3c5786415af46', 'request_id': 'req-8b27dd11-7d11-4fdc-96ff-c744ad54004c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.220521 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8b27dd11-7d11-4fdc-96ff-c744ad54004c admin admin] The token cache is being invalidate because project 22ca79a036874f679df3c5786415af46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:37.220813 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8b27dd11-7d11-4fdc-96ff-c744ad54004c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.221453 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8b27dd11-7d11-4fdc-96ff-c744ad54004c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.222952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 989/1949] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:37 2022] DELETE /identity/v3/projects/22ca79a036874f679df3c5786415af46 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:37.226861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4027af-9a9b-4707-a4e0-3999f3fc71f9 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:37.228333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7a4027af-9a9b-4707-a4e0-3999f3fc71f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:37.241303 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4027af-9a9b-4707-a4e0-3999f3fc71f9 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:37.299365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56d5c6-ed07-4a6f-b3a0-ef09edfe4ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:37.299944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56d5c6-ed07-4a6f-b3a0-ef09edfe4ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:37.300224 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56d5c6-ed07-4a6f-b3a0-ef09edfe4ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:37.300495 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56d5c6-ed07-4a6f-b3a0-ef09edfe4ce3 admin admin] PATH_INFO: `/v3/users/bb22418d4e6343f7bc2857c36be1e6a8` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:37.301617 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56d5c6-ed07-4a6f-b3a0-ef09edfe4ce3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb22418d4e6343f7bc2857c36be1e6a8)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:37.303568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56d5c6-ed07-4a6f-b3a0-ef09edfe4ce3 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:37.305886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:37.352379 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-de56d5c6-ed07-4a6f-b3a0-ef09edfe4ce3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb22418d4e6343f7bc2857c36be1e6a8'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.353026 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-de56d5c6-ed07-4a6f-b3a0-ef09edfe4ce3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb22418d4e6343f7bc2857c36be1e6a8'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.353630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-de56d5c6-ed07-4a6f-b3a0-ef09edfe4ce3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb22418d4e6343f7bc2857c36be1e6a8'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.360522 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-de56d5c6-ed07-4a6f-b3a0-ef09edfe4ce3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb22418d4e6343f7bc2857c36be1e6a8'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.367231 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-de56d5c6-ed07-4a6f-b3a0-ef09edfe4ce3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb22418d4e6343f7bc2857c36be1e6a8'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.384677 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 961/1950] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:37 2022] DELETE /identity/v3/users/bb22418d4e6343f7bc2857c36be1e6a8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:37.566255 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27dd11-7d11-4fdc-96ff-c744ad54004c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:37.567862 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8b27dd11-7d11-4fdc-96ff-c744ad54004c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:37.587280 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27dd11-7d11-4fdc-96ff-c744ad54004c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:37.662984 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30ddee-e316-432d-a8d8-403756df10ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:37.663664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30ddee-e316-432d-a8d8-403756df10ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:37.663954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30ddee-e316-432d-a8d8-403756df10ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:37.664325 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30ddee-e316-432d-a8d8-403756df10ed admin admin] PATH_INFO: `/v3/projects/8cbee61459cb40b1b8453beb77eac4c8` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:37.665609 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30ddee-e316-432d-a8d8-403756df10ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cbee61459cb40b1b8453beb77eac4c8)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:37.667442 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30ddee-e316-432d-a8d8-403756df10ed admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:37.729558 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ea30ddee-e316-432d-a8d8-403756df10ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cbee61459cb40b1b8453beb77eac4c8', 'request_id': 'req-ea30ddee-e316-432d-a8d8-403756df10ed'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.739104 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ea30ddee-e316-432d-a8d8-403756df10ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cbee61459cb40b1b8453beb77eac4c8', 'request_id': 'req-ea30ddee-e316-432d-a8d8-403756df10ed'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.748528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ea30ddee-e316-432d-a8d8-403756df10ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cbee61459cb40b1b8453beb77eac4c8', 'request_id': 'req-ea30ddee-e316-432d-a8d8-403756df10ed'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.762092 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ea30ddee-e316-432d-a8d8-403756df10ed admin admin] The token cache is being invalidate because project 8cbee61459cb40b1b8453beb77eac4c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:37.762486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ea30ddee-e316-432d-a8d8-403756df10ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.763217 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ea30ddee-e316-432d-a8d8-403756df10ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:37.765321 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 990/1951] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:37 2022] DELETE /identity/v3/projects/8cbee61459cb40b1b8453beb77eac4c8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:37.769950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b90340-556c-4ea6-9c85-8485b5804cc2 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:37.770239 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b90340-556c-4ea6-9c85-8485b5804cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:37.770521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b90340-556c-4ea6-9c85-8485b5804cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:37.792668 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-66b90340-556c-4ea6-9c85-8485b5804cc2 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:37.793910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-66b90340-556c-4ea6-9c85-8485b5804cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:37.860291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 962/1952] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:37.868173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec2dba-cf53-4808-9eca-6639222238f9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:37.868470 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec2dba-cf53-4808-9eca-6639222238f9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:37.868755 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec2dba-cf53-4808-9eca-6639222238f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:37.888480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-51ec2dba-cf53-4808-9eca-6639222238f9 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:37.890229 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-51ec2dba-cf53-4808-9eca-6639222238f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:37.898157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 991/1953] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:37.902407 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b90340-556c-4ea6-9c85-8485b5804cc2 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:37.912819 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b90340-556c-4ea6-9c85-8485b5804cc2 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:37.917517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fe453-a59b-4b42-befc-acc2c7ecd50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:37.918117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe453-a59b-4b42-befc-acc2c7ecd50c admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:37.918392 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe453-a59b-4b42-befc-acc2c7ecd50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:37.918659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe453-a59b-4b42-befc-acc2c7ecd50c admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:37.919330 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fe453-a59b-4b42-befc-acc2c7ecd50c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:37.919642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fe453-a59b-4b42-befc-acc2c7ecd50c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:37.921941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fe453-a59b-4b42-befc-acc2c7ecd50c admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:37.933238 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 963/1954] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:37.938353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec2dba-cf53-4808-9eca-6639222238f9 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:37.947677 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec2dba-cf53-4808-9eca-6639222238f9 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:37.952702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd977c0-5ffd-4a43-bf8d-322b10e6c975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:37.953302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd977c0-5ffd-4a43-bf8d-322b10e6c975 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:37.953584 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd977c0-5ffd-4a43-bf8d-322b10e6c975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:37.953859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd977c0-5ffd-4a43-bf8d-322b10e6c975 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:37.954525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd977c0-5ffd-4a43-bf8d-322b10e6c975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:37.956462 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd977c0-5ffd-4a43-bf8d-322b10e6c975 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:37.988093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 992/1955] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:37 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:37.989293 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fe453-a59b-4b42-befc-acc2c7ecd50c admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.000394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fe453-a59b-4b42-befc-acc2c7ecd50c admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.056292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acede124-dc4b-44c8-b869-164ca8f092fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.056898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acede124-dc4b-44c8-b869-164ca8f092fc admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.057182 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acede124-dc4b-44c8-b869-164ca8f092fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.057589 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acede124-dc4b-44c8-b869-164ca8f092fc admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.058271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acede124-dc4b-44c8-b869-164ca8f092fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.060182 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acede124-dc4b-44c8-b869-164ca8f092fc admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.076654 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:38.085746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 964/1956] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:37 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:38.087062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd977c0-5ffd-4a43-bf8d-322b10e6c975 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.098156 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd977c0-5ffd-4a43-bf8d-322b10e6c975 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.102827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d833f-a8a6-4e01-9146-90510de87235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.103426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d833f-a8a6-4e01-9146-90510de87235 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.103743 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d833f-a8a6-4e01-9146-90510de87235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.104021 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d833f-a8a6-4e01-9146-90510de87235 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.104700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d833f-a8a6-4e01-9146-90510de87235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.106311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d833f-a8a6-4e01-9146-90510de87235 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.107712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:38.121189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 993/1957] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:38.125268 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acede124-dc4b-44c8-b869-164ca8f092fc admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.136043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acede124-dc4b-44c8-b869-164ca8f092fc admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.141479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53510973-6350-4aea-867d-d23fb1478b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.142614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53510973-6350-4aea-867d-d23fb1478b32 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.143040 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53510973-6350-4aea-867d-d23fb1478b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.143512 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53510973-6350-4aea-867d-d23fb1478b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.144337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53510973-6350-4aea-867d-d23fb1478b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.146293 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53510973-6350-4aea-867d-d23fb1478b32 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.160471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 965/1958] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:38.165354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d833f-a8a6-4e01-9146-90510de87235 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.176711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d833f-a8a6-4e01-9146-90510de87235 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.187134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e7172-a48e-4a32-92aa-0fa742dfa28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.187847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e7172-a48e-4a32-92aa-0fa742dfa28a admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.188133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e7172-a48e-4a32-92aa-0fa742dfa28a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.188412 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e7172-a48e-4a32-92aa-0fa742dfa28a admin admin] PATH_INFO: `/v3/projects/7dd5f1c8bfaa457085f5458126489a1a/users/a9b8068041ed4006bd68b77e5c621390/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.215199 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e7172-a48e-4a32-92aa-0fa742dfa28a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dd5f1c8bfaa457085f5458126489a1a, user_id=a9b8068041ed4006bd68b77e5c621390, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.219101 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e7172-a48e-4a32-92aa-0fa742dfa28a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.236283 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 994/1959] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:38 2022] PUT /identity/v3/projects/7dd5f1c8bfaa457085f5458126489a1a/users/a9b8068041ed4006bd68b77e5c621390/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:38.247769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb9852-6500-437d-a02f-699ac8c79b22 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.248217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb9852-6500-437d-a02f-699ac8c79b22 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.249139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb9852-6500-437d-a02f-699ac8c79b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.279664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-e7cb9852-6500-437d-a02f-699ac8c79b22 None None] MFA Rules not processed for user `a9b8068041ed4006bd68b77e5c621390`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:38.281573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e7cb9852-6500-437d-a02f-699ac8c79b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:38.374856 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 966/1960] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:38.379909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e7172-a48e-4a32-92aa-0fa742dfa28a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.466464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e7172-a48e-4a32-92aa-0fa742dfa28a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.471869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383e4ad-192f-4191-9916-93744b7b250e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.472480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383e4ad-192f-4191-9916-93744b7b250e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.472761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383e4ad-192f-4191-9916-93744b7b250e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.473621 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9383e4ad-192f-4191-9916-93744b7b250e admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.474254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383e4ad-192f-4191-9916-93744b7b250e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.480373 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9383e4ad-192f-4191-9916-93744b7b250e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.512276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 995/1961] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:38 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:38.516758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb9852-6500-437d-a02f-699ac8c79b22 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.534766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb9852-6500-437d-a02f-699ac8c79b22 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.599005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf2cac-f23d-4266-8867-c9faa40d92eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.599534 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf2cac-f23d-4266-8867-c9faa40d92eb admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.599833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf2cac-f23d-4266-8867-c9faa40d92eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.600115 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf2cac-f23d-4266-8867-c9faa40d92eb admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.600752 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf2cac-f23d-4266-8867-c9faa40d92eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.602662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf2cac-f23d-4266-8867-c9faa40d92eb admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.628396 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 967/1962] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:38 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:38.629865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383e4ad-192f-4191-9916-93744b7b250e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.641302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9383e4ad-192f-4191-9916-93744b7b250e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.646006 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cc386-f71b-4a31-98c6-1d30af12bfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.646621 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cc386-f71b-4a31-98c6-1d30af12bfd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.646901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cc386-f71b-4a31-98c6-1d30af12bfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.647177 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728cc386-f71b-4a31-98c6-1d30af12bfd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.647859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cc386-f71b-4a31-98c6-1d30af12bfd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.649483 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728cc386-f71b-4a31-98c6-1d30af12bfd1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.661453 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 996/1963] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:38.667011 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf2cac-f23d-4266-8867-c9faa40d92eb admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.678350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf2cac-f23d-4266-8867-c9faa40d92eb admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.683180 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae2621-5f9e-4aca-bb82-ea9f1c99fa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.683819 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae2621-5f9e-4aca-bb82-ea9f1c99fa62 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.684099 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae2621-5f9e-4aca-bb82-ea9f1c99fa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.684371 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae2621-5f9e-4aca-bb82-ea9f1c99fa62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.685015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae2621-5f9e-4aca-bb82-ea9f1c99fa62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.687384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae2621-5f9e-4aca-bb82-ea9f1c99fa62 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.698079 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 968/1964] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:38.702048 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cc386-f71b-4a31-98c6-1d30af12bfd1 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.712650 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728cc386-f71b-4a31-98c6-1d30af12bfd1 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.717508 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c7926-aed5-4414-8580-e1a463fb6ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.718808 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c7926-aed5-4414-8580-e1a463fb6ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.718808 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c7926-aed5-4414-8580-e1a463fb6ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.718996 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c7926-aed5-4414-8580-e1a463fb6ee8 admin admin] PATH_INFO: `/v3/projects/6c5bbf74b53b4a4eabf15aff8a61f0ac/users/f3b0c6781b0e4d1f92bc06ec80e43a49/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.738499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c7926-aed5-4414-8580-e1a463fb6ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c5bbf74b53b4a4eabf15aff8a61f0ac, user_id=f3b0c6781b0e4d1f92bc06ec80e43a49, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.741581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c7926-aed5-4414-8580-e1a463fb6ee8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:38.758701 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 997/1965] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:38 2022] PUT /identity/v3/projects/6c5bbf74b53b4a4eabf15aff8a61f0ac/users/f3b0c6781b0e4d1f92bc06ec80e43a49/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:38.768896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71272483-bca6-43ec-8650-d94ba28eb539 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.769188 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71272483-bca6-43ec-8650-d94ba28eb539 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.769468 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71272483-bca6-43ec-8650-d94ba28eb539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.799250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-71272483-bca6-43ec-8650-d94ba28eb539 None None] MFA Rules not processed for user `f3b0c6781b0e4d1f92bc06ec80e43a49`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:38.800515 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-71272483-bca6-43ec-8650-d94ba28eb539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:38.856142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:38.898334 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 969/1966] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:38.903681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c7926-aed5-4414-8580-e1a463fb6ee8 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:38.988311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c7926-aed5-4414-8580-e1a463fb6ee8 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:38.993583 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df666a4-d084-41ac-b09a-c9596abe5780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:38.994230 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df666a4-d084-41ac-b09a-c9596abe5780 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:38.994513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df666a4-d084-41ac-b09a-c9596abe5780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:38.994788 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df666a4-d084-41ac-b09a-c9596abe5780 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:38.995419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df666a4-d084-41ac-b09a-c9596abe5780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:38.997435 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df666a4-d084-41ac-b09a-c9596abe5780 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.032905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 998/1967] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:38 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:39.036759 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71272483-bca6-43ec-8650-d94ba28eb539 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.050204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71272483-bca6-43ec-8650-d94ba28eb539 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.116953 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92405260-6e43-48a0-87e5-307b1a3419aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.117682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92405260-6e43-48a0-87e5-307b1a3419aa admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.118020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92405260-6e43-48a0-87e5-307b1a3419aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.118411 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92405260-6e43-48a0-87e5-307b1a3419aa admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.119133 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92405260-6e43-48a0-87e5-307b1a3419aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.121224 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92405260-6e43-48a0-87e5-307b1a3419aa admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.175418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 970/1968] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:39 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:39.179575 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df666a4-d084-41ac-b09a-c9596abe5780 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.191380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df666a4-d084-41ac-b09a-c9596abe5780 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.196394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e768bc3-d466-4e7c-9cca-9e6d4c022ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.197042 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e768bc3-d466-4e7c-9cca-9e6d4c022ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.197309 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e768bc3-d466-4e7c-9cca-9e6d4c022ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.197585 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e768bc3-d466-4e7c-9cca-9e6d4c022ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.198258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e768bc3-d466-4e7c-9cca-9e6d4c022ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.199977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e768bc3-d466-4e7c-9cca-9e6d4c022ae0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.202685 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:39.218946 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 999/1969] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:39.223152 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92405260-6e43-48a0-87e5-307b1a3419aa admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.238387 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92405260-6e43-48a0-87e5-307b1a3419aa admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.244650 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d215f42-f080-40d6-a78f-8a34f19b8d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.245350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d215f42-f080-40d6-a78f-8a34f19b8d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.245703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d215f42-f080-40d6-a78f-8a34f19b8d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.246058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d215f42-f080-40d6-a78f-8a34f19b8d49 admin admin] PATH_INFO: `/v3/projects/19b64cfc511241f7a7198e1df82ef84c/users/e424cff2ad614be986215629ee39376f/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.270499 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d215f42-f080-40d6-a78f-8a34f19b8d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b64cfc511241f7a7198e1df82ef84c, user_id=e424cff2ad614be986215629ee39376f, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.275758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d215f42-f080-40d6-a78f-8a34f19b8d49 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.302330 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 971/1970] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:39 2022] PUT /identity/v3/projects/19b64cfc511241f7a7198e1df82ef84c/users/e424cff2ad614be986215629ee39376f/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:39.303787 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e768bc3-d466-4e7c-9cca-9e6d4c022ae0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.315483 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e768bc3-d466-4e7c-9cca-9e6d4c022ae0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.419100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92327503-1000-42c0-b759-34ffdeeaea64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.419782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92327503-1000-42c0-b759-34ffdeeaea64 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.420095 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92327503-1000-42c0-b759-34ffdeeaea64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.421015 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92327503-1000-42c0-b759-34ffdeeaea64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.421671 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92327503-1000-42c0-b759-34ffdeeaea64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.423518 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92327503-1000-42c0-b759-34ffdeeaea64 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.439358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d215f42-f080-40d6-a78f-8a34f19b8d49 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.450611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1000/1971] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:39.451129 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d215f42-f080-40d6-a78f-8a34f19b8d49 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.463773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37222a96-7e35-4cca-a1e5-9ccbafc2d2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.465116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37222a96-7e35-4cca-a1e5-9ccbafc2d2d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.465708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37222a96-7e35-4cca-a1e5-9ccbafc2d2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.466303 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37222a96-7e35-4cca-a1e5-9ccbafc2d2d2 admin admin] PATH_INFO: `/v3/projects/19b64cfc511241f7a7198e1df82ef84c/users/e424cff2ad614be986215629ee39376f/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.469590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37222a96-7e35-4cca-a1e5-9ccbafc2d2d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b64cfc511241f7a7198e1df82ef84c, user_id=e424cff2ad614be986215629ee39376f, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.472944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37222a96-7e35-4cca-a1e5-9ccbafc2d2d2 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.492124 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 972/1972] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:39 2022] PUT /identity/v3/projects/19b64cfc511241f7a7198e1df82ef84c/users/e424cff2ad614be986215629ee39376f/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:39.497950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5ae61-36c3-4d85-b253-58bced1da6f7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.498233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5ae61-36c3-4d85-b253-58bced1da6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.498538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5ae61-36c3-4d85-b253-58bced1da6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.535097 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-37f5ae61-36c3-4d85-b253-58bced1da6f7 None None] MFA Rules not processed for user `e424cff2ad614be986215629ee39376f`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:39.536367 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-37f5ae61-36c3-4d85-b253-58bced1da6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:39.625144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1001/1973] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:39.641622 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37222a96-7e35-4cca-a1e5-9ccbafc2d2d2 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:39.643662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-37222a96-7e35-4cca-a1e5-9ccbafc2d2d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:39.728043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37222a96-7e35-4cca-a1e5-9ccbafc2d2d2 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:39.732524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515ee87-d89c-440f-ab76-8268e1ad75f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:39.733972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515ee87-d89c-440f-ab76-8268e1ad75f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:39.739139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515ee87-d89c-440f-ab76-8268e1ad75f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:39.739431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515ee87-d89c-440f-ab76-8268e1ad75f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:39.796220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:39.806296 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515ee87-d89c-440f-ab76-8268e1ad75f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:39.807778 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515ee87-d89c-440f-ab76-8268e1ad75f8 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:39.817136 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 973/1974] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:44.505611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5ae61-36c3-4d85-b253-58bced1da6f7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:44.515863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5ae61-36c3-4d85-b253-58bced1da6f7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:44.519783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0a82e-3658-434b-aca1-8f4e1125eb4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:44.520390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0a82e-3658-434b-aca1-8f4e1125eb4d service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:44.520675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0a82e-3658-434b-aca1-8f4e1125eb4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:44.520951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0a82e-3658-434b-aca1-8f4e1125eb4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:44.583202 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0a82e-3658-434b-aca1-8f4e1125eb4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:44.584744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0a82e-3658-434b-aca1-8f4e1125eb4d service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:44.593673 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1002/1975] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:45.176325 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515ee87-d89c-440f-ab76-8268e1ad75f8 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:45.194644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515ee87-d89c-440f-ab76-8268e1ad75f8 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:45.198876 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997859a5-cf6c-4914-8ac5-f1bd8edd848a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:45.199555 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997859a5-cf6c-4914-8ac5-f1bd8edd848a service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:45.199838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997859a5-cf6c-4914-8ac5-f1bd8edd848a service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:45.200114 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997859a5-cf6c-4914-8ac5-f1bd8edd848a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:45.210297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997859a5-cf6c-4914-8ac5-f1bd8edd848a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:45.211757 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997859a5-cf6c-4914-8ac5-f1bd8edd848a service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:45.223748 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 974/1976] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:46.139913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0a82e-3658-434b-aca1-8f4e1125eb4d service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:46.141611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2ba0a82e-3658-434b-aca1-8f4e1125eb4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:46.164479 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:46.215422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0a82e-3658-434b-aca1-8f4e1125eb4d service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:46.220801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebffc4-ec1e-4b8f-a5df-3552cb10dec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:46.221473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebffc4-ec1e-4b8f-a5df-3552cb10dec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:46.221764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebffc4-ec1e-4b8f-a5df-3552cb10dec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:46.222045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebffc4-ec1e-4b8f-a5df-3552cb10dec0 admin admin] PATH_INFO: `/v3/users/39bb5639c8284c69809cc9f206bbbfb7` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:46.223193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daebffc4-ec1e-4b8f-a5df-3552cb10dec0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39bb5639c8284c69809cc9f206bbbfb7)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:46.225180 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daebffc4-ec1e-4b8f-a5df-3552cb10dec0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:46.281800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-daebffc4-ec1e-4b8f-a5df-3552cb10dec0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39bb5639c8284c69809cc9f206bbbfb7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:46.282603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-daebffc4-ec1e-4b8f-a5df-3552cb10dec0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39bb5639c8284c69809cc9f206bbbfb7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:46.291917 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-daebffc4-ec1e-4b8f-a5df-3552cb10dec0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39bb5639c8284c69809cc9f206bbbfb7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:46.300026 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-daebffc4-ec1e-4b8f-a5df-3552cb10dec0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39bb5639c8284c69809cc9f206bbbfb7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:46.316912 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-daebffc4-ec1e-4b8f-a5df-3552cb10dec0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39bb5639c8284c69809cc9f206bbbfb7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:46.324298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1003/1977] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:46 2022] DELETE /identity/v3/users/39bb5639c8284c69809cc9f206bbbfb7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:46.331209 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7b022-2da1-4483-8b92-45426f0113d4 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:46.331837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7b022-2da1-4483-8b92-45426f0113d4 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:46.331837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7b022-2da1-4483-8b92-45426f0113d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:46.358445 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-b6e7b022-2da1-4483-8b92-45426f0113d4 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:46.360913 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b6e7b022-2da1-4483-8b92-45426f0113d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:46.461607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 975/1978] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:46.472745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebffc4-ec1e-4b8f-a5df-3552cb10dec0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:46.474701 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-daebffc4-ec1e-4b8f-a5df-3552cb10dec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:46.546445 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebffc4-ec1e-4b8f-a5df-3552cb10dec0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:46.551120 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73258c53-73eb-4ebb-a88f-e0d34b65375b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:46.551845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73258c53-73eb-4ebb-a88f-e0d34b65375b service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:46.552131 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73258c53-73eb-4ebb-a88f-e0d34b65375b service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:46.552405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73258c53-73eb-4ebb-a88f-e0d34b65375b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:46.563915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73258c53-73eb-4ebb-a88f-e0d34b65375b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:46.565539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73258c53-73eb-4ebb-a88f-e0d34b65375b service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:46.583869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1004/1979] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:47.021808 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7b022-2da1-4483-8b92-45426f0113d4 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:47.023255 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b6e7b022-2da1-4483-8b92-45426f0113d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:47.040543 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7b022-2da1-4483-8b92-45426f0113d4 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:47.045727 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:47.047228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:47.048040 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:47.048447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca admin admin] PATH_INFO: `/v3/projects/1360541ac75e4cb6b58b607698fc4711` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:47.050267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=1360541ac75e4cb6b58b607698fc4711)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:47.052684 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:47.101771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1360541ac75e4cb6b58b607698fc4711', 'request_id': 'req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.107886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1360541ac75e4cb6b58b607698fc4711', 'request_id': 'req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.116089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1360541ac75e4cb6b58b607698fc4711', 'request_id': 'req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.128724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca admin admin] The token cache is being invalidate because project 1360541ac75e4cb6b58b607698fc4711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:47.128927 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.130566 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.132573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 976/1980] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:47 2022] DELETE /identity/v3/projects/1360541ac75e4cb6b58b607698fc4711 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:47.136583 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73258c53-73eb-4ebb-a88f-e0d34b65375b service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:47.146907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73258c53-73eb-4ebb-a88f-e0d34b65375b service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:47.191746 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:47.215902 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a94b4-f782-4519-bbb0-b5785d4ef380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:47.216549 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a94b4-f782-4519-bbb0-b5785d4ef380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:47.216827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a94b4-f782-4519-bbb0-b5785d4ef380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:47.217101 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a94b4-f782-4519-bbb0-b5785d4ef380 admin admin] PATH_INFO: `/v3/users/7ffa4f761096401881b2765b266ef432` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:47.218249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a94b4-f782-4519-bbb0-b5785d4ef380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ffa4f761096401881b2765b266ef432)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:47.221164 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a94b4-f782-4519-bbb0-b5785d4ef380 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:47.279749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b51a94b4-f782-4519-bbb0-b5785d4ef380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ffa4f761096401881b2765b266ef432'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.280289 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b51a94b4-f782-4519-bbb0-b5785d4ef380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ffa4f761096401881b2765b266ef432'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.288365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b51a94b4-f782-4519-bbb0-b5785d4ef380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ffa4f761096401881b2765b266ef432'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.299113 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b51a94b4-f782-4519-bbb0-b5785d4ef380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ffa4f761096401881b2765b266ef432'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.313878 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b51a94b4-f782-4519-bbb0-b5785d4ef380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ffa4f761096401881b2765b266ef432'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:47.320425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1005/1981] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:47 2022] DELETE /identity/v3/users/7ffa4f761096401881b2765b266ef432 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:47.980719 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:47.982182 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:47.994994 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79eb05f-1bd9-4137-8bc3-ca628057e0ca admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:48.059083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040be63d-f863-437f-ba1c-23d798df8724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:48.059757 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040be63d-f863-437f-ba1c-23d798df8724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.060047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040be63d-f863-437f-ba1c-23d798df8724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.060324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040be63d-f863-437f-ba1c-23d798df8724 admin admin] PATH_INFO: `/v3/projects/baf6370948eb4bf7a437d8127b23c0ae` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.061464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040be63d-f863-437f-ba1c-23d798df8724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baf6370948eb4bf7a437d8127b23c0ae)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:48.063945 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040be63d-f863-437f-ba1c-23d798df8724 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:48.068439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:48.117143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-040be63d-f863-437f-ba1c-23d798df8724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baf6370948eb4bf7a437d8127b23c0ae', 'request_id': 'req-040be63d-f863-437f-ba1c-23d798df8724'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.122704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-040be63d-f863-437f-ba1c-23d798df8724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baf6370948eb4bf7a437d8127b23c0ae', 'request_id': 'req-040be63d-f863-437f-ba1c-23d798df8724'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.131166 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-040be63d-f863-437f-ba1c-23d798df8724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baf6370948eb4bf7a437d8127b23c0ae', 'request_id': 'req-040be63d-f863-437f-ba1c-23d798df8724'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.145036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-040be63d-f863-437f-ba1c-23d798df8724 admin admin] The token cache is being invalidate because project baf6370948eb4bf7a437d8127b23c0ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:48.145344 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-040be63d-f863-437f-ba1c-23d798df8724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.146066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-040be63d-f863-437f-ba1c-23d798df8724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:48.148218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 977/1982] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:47 2022] DELETE /identity/v3/projects/baf6370948eb4bf7a437d8127b23c0ae => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:48.153178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d12ea-aa96-4c7b-b458-5c1e79f4c5ee None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.153809 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d12ea-aa96-4c7b-b458-5c1e79f4c5ee None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.153809 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d12ea-aa96-4c7b-b458-5c1e79f4c5ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.177029 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-986d12ea-aa96-4c7b-b458-5c1e79f4c5ee None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:48.178532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-986d12ea-aa96-4c7b-b458-5c1e79f4c5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:48.253735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1006/1983] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:48.262025 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539b8c6-a37c-4282-a110-6af337908a7b None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.262381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539b8c6-a37c-4282-a110-6af337908a7b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.262747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539b8c6-a37c-4282-a110-6af337908a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.285448 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-4539b8c6-a37c-4282-a110-6af337908a7b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:48.286734 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-4539b8c6-a37c-4282-a110-6af337908a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:48.294854 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 978/1984] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:48.299571 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d12ea-aa96-4c7b-b458-5c1e79f4c5ee None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:48.309528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d12ea-aa96-4c7b-b458-5c1e79f4c5ee None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:48.316035 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c834c6-c9c4-475f-aa8e-56f406a3368c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:48.316642 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c834c6-c9c4-475f-aa8e-56f406a3368c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.316918 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c834c6-c9c4-475f-aa8e-56f406a3368c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.317184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c834c6-c9c4-475f-aa8e-56f406a3368c admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.317832 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c834c6-c9c4-475f-aa8e-56f406a3368c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:48.318134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c834c6-c9c4-475f-aa8e-56f406a3368c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:48.319750 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c834c6-c9c4-475f-aa8e-56f406a3368c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:48.332702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1007/1985] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:48.336890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539b8c6-a37c-4282-a110-6af337908a7b None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:48.346198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539b8c6-a37c-4282-a110-6af337908a7b None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:48.350987 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd01ac3-e6cf-466a-a665-3850f6643a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:48.351586 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd01ac3-e6cf-466a-a665-3850f6643a70 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.351904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd01ac3-e6cf-466a-a665-3850f6643a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.352175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd01ac3-e6cf-466a-a665-3850f6643a70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.352794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd01ac3-e6cf-466a-a665-3850f6643a70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:48.354675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd01ac3-e6cf-466a-a665-3850f6643a70 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:48.384348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 979/1986] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:48 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:48.388472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c834c6-c9c4-475f-aa8e-56f406a3368c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:48.399359 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c834c6-c9c4-475f-aa8e-56f406a3368c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:48.448425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:48.467353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3b1fb-23ee-4e64-814f-c7c2ba1ce6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:48.467999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3b1fb-23ee-4e64-814f-c7c2ba1ce6e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.468323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3b1fb-23ee-4e64-814f-c7c2ba1ce6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.469191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3b1fb-23ee-4e64-814f-c7c2ba1ce6e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.470109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3b1fb-23ee-4e64-814f-c7c2ba1ce6e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:48.472062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3b1fb-23ee-4e64-814f-c7c2ba1ce6e3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:48.496332 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1008/1987] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:48 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:48.497520 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd01ac3-e6cf-466a-a665-3850f6643a70 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:48.508619 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd01ac3-e6cf-466a-a665-3850f6643a70 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:48.513456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485243a4-fda8-4fe3-a344-4013354d051d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:48.514063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485243a4-fda8-4fe3-a344-4013354d051d admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.514348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485243a4-fda8-4fe3-a344-4013354d051d admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.514621 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485243a4-fda8-4fe3-a344-4013354d051d admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.515799 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485243a4-fda8-4fe3-a344-4013354d051d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:48.517581 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485243a4-fda8-4fe3-a344-4013354d051d admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:48.531412 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 980/1988] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:48.533282 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3b1fb-23ee-4e64-814f-c7c2ba1ce6e3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:48.543670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3b1fb-23ee-4e64-814f-c7c2ba1ce6e3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:48.551369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc9a9d-a259-4ab6-a7e1-6e1991c2450a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:48.552088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc9a9d-a259-4ab6-a7e1-6e1991c2450a admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.552375 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc9a9d-a259-4ab6-a7e1-6e1991c2450a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.552652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc9a9d-a259-4ab6-a7e1-6e1991c2450a admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.553306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc9a9d-a259-4ab6-a7e1-6e1991c2450a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:48.554920 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc9a9d-a259-4ab6-a7e1-6e1991c2450a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:48.567098 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1009/1989] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:48.571797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485243a4-fda8-4fe3-a344-4013354d051d admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:48.583208 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485243a4-fda8-4fe3-a344-4013354d051d admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:48.594544 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b756c32-c469-4007-b634-7c013c693d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:48.595325 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b756c32-c469-4007-b634-7c013c693d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.595738 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b756c32-c469-4007-b634-7c013c693d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.596084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b756c32-c469-4007-b634-7c013c693d4f admin admin] PATH_INFO: `/v3/projects/737554c4d002424a9291b04f56fdf8aa/users/a6f48aa2e6934f28b67cc9d090a79c08/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.613197 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b756c32-c469-4007-b634-7c013c693d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=737554c4d002424a9291b04f56fdf8aa, user_id=a6f48aa2e6934f28b67cc9d090a79c08, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:48.616086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b756c32-c469-4007-b634-7c013c693d4f admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:48.632961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 981/1990] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:48 2022] PUT /identity/v3/projects/737554c4d002424a9291b04f56fdf8aa/users/a6f48aa2e6934f28b67cc9d090a79c08/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:48.646270 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9167fa-622d-4e9d-a7e3-9bdb2e2d1806 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.646564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9167fa-622d-4e9d-a7e3-9bdb2e2d1806 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.646850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9167fa-622d-4e9d-a7e3-9bdb2e2d1806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.675282 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9a9167fa-622d-4e9d-a7e3-9bdb2e2d1806 None None] MFA Rules not processed for user `a6f48aa2e6934f28b67cc9d090a79c08`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:48.676743 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9a9167fa-622d-4e9d-a7e3-9bdb2e2d1806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:48.758978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1010/1991] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:48.764036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b756c32-c469-4007-b634-7c013c693d4f admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:48.853599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b756c32-c469-4007-b634-7c013c693d4f admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:48.858669 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24847061-a2e9-4029-a18d-97a16ab41181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:48.859334 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24847061-a2e9-4029-a18d-97a16ab41181 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:48.859845 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24847061-a2e9-4029-a18d-97a16ab41181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:48.859979 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24847061-a2e9-4029-a18d-97a16ab41181 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:48.860626 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24847061-a2e9-4029-a18d-97a16ab41181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:48.862635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24847061-a2e9-4029-a18d-97a16ab41181 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:48.910639 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 982/1992] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:48 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:48.911773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9167fa-622d-4e9d-a7e3-9bdb2e2d1806 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:48.926202 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9167fa-622d-4e9d-a7e3-9bdb2e2d1806 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.001885 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce716fcc-f58a-4760-83a9-783e19ccf02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.002395 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce716fcc-f58a-4760-83a9-783e19ccf02e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.003246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce716fcc-f58a-4760-83a9-783e19ccf02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.003246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce716fcc-f58a-4760-83a9-783e19ccf02e admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.003840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce716fcc-f58a-4760-83a9-783e19ccf02e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.005916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce716fcc-f58a-4760-83a9-783e19ccf02e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.030821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1011/1993] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:48 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:49.034889 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24847061-a2e9-4029-a18d-97a16ab41181 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.045565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24847061-a2e9-4029-a18d-97a16ab41181 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.051125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ce2a9-c2c3-4d2d-8352-a8a01b123c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.051769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ce2a9-c2c3-4d2d-8352-a8a01b123c62 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.052055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ce2a9-c2c3-4d2d-8352-a8a01b123c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.052332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73ce2a9-c2c3-4d2d-8352-a8a01b123c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.052991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ce2a9-c2c3-4d2d-8352-a8a01b123c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.054601 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73ce2a9-c2c3-4d2d-8352-a8a01b123c62 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.073663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 983/1994] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:49.074277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce716fcc-f58a-4760-83a9-783e19ccf02e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.087821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce716fcc-f58a-4760-83a9-783e19ccf02e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.092973 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d25c09-fead-4396-b437-317d62202107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.093598 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d25c09-fead-4396-b437-317d62202107 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.093881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d25c09-fead-4396-b437-317d62202107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.094164 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d25c09-fead-4396-b437-317d62202107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.094819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d25c09-fead-4396-b437-317d62202107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.096480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d25c09-fead-4396-b437-317d62202107 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.110062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1012/1995] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:49.114501 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ce2a9-c2c3-4d2d-8352-a8a01b123c62 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.120402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:49.128195 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73ce2a9-c2c3-4d2d-8352-a8a01b123c62 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.135040 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c30605-b872-47b4-b2b2-db85730c3e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.136414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c30605-b872-47b4-b2b2-db85730c3e7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.136913 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c30605-b872-47b4-b2b2-db85730c3e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.137650 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c30605-b872-47b4-b2b2-db85730c3e7d admin admin] PATH_INFO: `/v3/projects/d13dfe16c971434193e467f46f1ecd10/users/938712f670b84a829d093a56f8b45bb9/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.165640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c30605-b872-47b4-b2b2-db85730c3e7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d13dfe16c971434193e467f46f1ecd10, user_id=938712f670b84a829d093a56f8b45bb9, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.168449 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c30605-b872-47b4-b2b2-db85730c3e7d admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.185631 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 984/1996] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:49 2022] PUT /identity/v3/projects/d13dfe16c971434193e467f46f1ecd10/users/938712f670b84a829d093a56f8b45bb9/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:49.197751 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd870806-aa3f-41f9-96a9-aee65d8212c7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.198148 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd870806-aa3f-41f9-96a9-aee65d8212c7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.198301 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd870806-aa3f-41f9-96a9-aee65d8212c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.229705 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-cd870806-aa3f-41f9-96a9-aee65d8212c7 None None] MFA Rules not processed for user `938712f670b84a829d093a56f8b45bb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:49.231496 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cd870806-aa3f-41f9-96a9-aee65d8212c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:49.260359 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:49.313487 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1013/1997] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:49.317689 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c30605-b872-47b4-b2b2-db85730c3e7d admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.421821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c30605-b872-47b4-b2b2-db85730c3e7d admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.426832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23fc10-f681-429c-b0cf-b0a140d3fd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.427475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23fc10-f681-429c-b0cf-b0a140d3fd51 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.427782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23fc10-f681-429c-b0cf-b0a140d3fd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.428086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23fc10-f681-429c-b0cf-b0a140d3fd51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.428714 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23fc10-f681-429c-b0cf-b0a140d3fd51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.430651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23fc10-f681-429c-b0cf-b0a140d3fd51 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.471724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 985/1998] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:49 2022] POST /identity/v3/projects => generated 385 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:49.475726 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd870806-aa3f-41f9-96a9-aee65d8212c7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.489554 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd870806-aa3f-41f9-96a9-aee65d8212c7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.491020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23fc10-f681-429c-b0cf-b0a140d3fd51 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.492923 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-ab23fc10-f681-429c-b0cf-b0a140d3fd51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:49.566982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23fc10-f681-429c-b0cf-b0a140d3fd51 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.571833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f754674-9647-4ba0-b6b8-3b3f9d957fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.572468 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f754674-9647-4ba0-b6b8-3b3f9d957fdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.572749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f754674-9647-4ba0-b6b8-3b3f9d957fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.573021 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f754674-9647-4ba0-b6b8-3b3f9d957fdf admin admin] PATH_INFO: `/v3/users/539573490e0d4e128caf93c51567ed79` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.573778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767eb45-d667-46d4-8c15-8d4f2156dd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.574175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f754674-9647-4ba0-b6b8-3b3f9d957fdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=539573490e0d4e128caf93c51567ed79)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.574782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767eb45-d667-46d4-8c15-8d4f2156dd50 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.575294 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767eb45-d667-46d4-8c15-8d4f2156dd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.575796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767eb45-d667-46d4-8c15-8d4f2156dd50 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.576227 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f754674-9647-4ba0-b6b8-3b3f9d957fdf admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.576646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767eb45-d667-46d4-8c15-8d4f2156dd50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.579903 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767eb45-d667-46d4-8c15-8d4f2156dd50 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.602470 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1014/1999] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:49 2022] POST /identity/v3/users => generated 299 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:49.606497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767eb45-d667-46d4-8c15-8d4f2156dd50 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.616759 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767eb45-d667-46d4-8c15-8d4f2156dd50 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.621504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466583da-af3f-499f-acad-bc55ac7195f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.622089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466583da-af3f-499f-acad-bc55ac7195f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.622374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466583da-af3f-499f-acad-bc55ac7195f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.622647 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466583da-af3f-499f-acad-bc55ac7195f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.623307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466583da-af3f-499f-acad-bc55ac7195f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.625339 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-0f754674-9647-4ba0-b6b8-3b3f9d957fdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '539573490e0d4e128caf93c51567ed79'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.625649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466583da-af3f-499f-acad-bc55ac7195f2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.626081 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-0f754674-9647-4ba0-b6b8-3b3f9d957fdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '539573490e0d4e128caf93c51567ed79'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.626720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-0f754674-9647-4ba0-b6b8-3b3f9d957fdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '539573490e0d4e128caf93c51567ed79'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.636885 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1015/2000] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:49.637095 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-0f754674-9647-4ba0-b6b8-3b3f9d957fdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '539573490e0d4e128caf93c51567ed79'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.644047 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466583da-af3f-499f-acad-bc55ac7195f2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.646267 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-466583da-af3f-499f-acad-bc55ac7195f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:49.646505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-0f754674-9647-4ba0-b6b8-3b3f9d957fdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '539573490e0d4e128caf93c51567ed79'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.649430 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:49.661872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466583da-af3f-499f-acad-bc55ac7195f2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.666591 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327db65-1119-4679-8fdf-9e4755dda0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.667193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327db65-1119-4679-8fdf-9e4755dda0bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.667477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327db65-1119-4679-8fdf-9e4755dda0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.667779 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327db65-1119-4679-8fdf-9e4755dda0bc admin admin] PATH_INFO: `/v3/projects/712cf6cde5e848d181d939a0a9d99dab/users/b98ef000d03847e1bced097aab816ed2/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.668165 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 986/2001] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:49 2022] DELETE /identity/v3/users/539573490e0d4e128caf93c51567ed79 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:49.683843 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327db65-1119-4679-8fdf-9e4755dda0bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=712cf6cde5e848d181d939a0a9d99dab, user_id=b98ef000d03847e1bced097aab816ed2, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.686616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327db65-1119-4679-8fdf-9e4755dda0bc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.700467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1016/2002] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:49 2022] PUT /identity/v3/projects/712cf6cde5e848d181d939a0a9d99dab/users/b98ef000d03847e1bced097aab816ed2/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:49.704988 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f754674-9647-4ba0-b6b8-3b3f9d957fdf admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.769897 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f754674-9647-4ba0-b6b8-3b3f9d957fdf admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.832463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea9aa3-82a3-4979-8479-5d19332616b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.833220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea9aa3-82a3-4979-8479-5d19332616b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.834007 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea9aa3-82a3-4979-8479-5d19332616b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.834182 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea9aa3-82a3-4979-8479-5d19332616b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.834918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea9aa3-82a3-4979-8479-5d19332616b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.836779 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea9aa3-82a3-4979-8479-5d19332616b2 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.849445 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327db65-1119-4679-8fdf-9e4755dda0bc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.849627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 987/2003] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:49.851243 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6327db65-1119-4679-8fdf-9e4755dda0bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:49.853494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea9aa3-82a3-4979-8479-5d19332616b2 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:49.868207 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea9aa3-82a3-4979-8479-5d19332616b2 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.872993 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297e89e-b697-4540-9705-04d2c31d39c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.873670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297e89e-b697-4540-9705-04d2c31d39c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.873958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297e89e-b697-4540-9705-04d2c31d39c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.874237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297e89e-b697-4540-9705-04d2c31d39c3 admin admin] PATH_INFO: `/v3/projects/712cf6cde5e848d181d939a0a9d99dab/users/b98ef000d03847e1bced097aab816ed2/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.879029 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327db65-1119-4679-8fdf-9e4755dda0bc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:49.881177 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297e89e-b697-4540-9705-04d2c31d39c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=712cf6cde5e848d181d939a0a9d99dab, user_id=b98ef000d03847e1bced097aab816ed2, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.887532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a73923-ce8b-424d-97a6-7deb14f917ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:49.888590 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a73923-ce8b-424d-97a6-7deb14f917ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.888862 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a73923-ce8b-424d-97a6-7deb14f917ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.889207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a73923-ce8b-424d-97a6-7deb14f917ae admin admin] PATH_INFO: `/v3/projects/2fd2c804427749c68ff038d019a1628b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.890656 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a73923-ce8b-424d-97a6-7deb14f917ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fd2c804427749c68ff038d019a1628b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:49.893024 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a73923-ce8b-424d-97a6-7deb14f917ae admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.899716 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297e89e-b697-4540-9705-04d2c31d39c3 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:49.913765 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 988/2004] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:49 2022] PUT /identity/v3/projects/712cf6cde5e848d181d939a0a9d99dab/users/b98ef000d03847e1bced097aab816ed2/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:49.925987 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8c21e-6057-4bf8-a9bd-719e642c74b2 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.926275 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8c21e-6057-4bf8-a9bd-719e642c74b2 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.926550 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8c21e-6057-4bf8-a9bd-719e642c74b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:49.956178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-11a73923-ce8b-424d-97a6-7deb14f917ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fd2c804427749c68ff038d019a1628b', 'request_id': 'req-11a73923-ce8b-424d-97a6-7deb14f917ae'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.961974 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-11a73923-ce8b-424d-97a6-7deb14f917ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fd2c804427749c68ff038d019a1628b', 'request_id': 'req-11a73923-ce8b-424d-97a6-7deb14f917ae'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.962491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-b2d8c21e-6057-4bf8-a9bd-719e642c74b2 None None] MFA Rules not processed for user `b98ef000d03847e1bced097aab816ed2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:49.963735 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b2d8c21e-6057-4bf8-a9bd-719e642c74b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:49.970439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-11a73923-ce8b-424d-97a6-7deb14f917ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fd2c804427749c68ff038d019a1628b', 'request_id': 'req-11a73923-ce8b-424d-97a6-7deb14f917ae'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.979686 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:49.986626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-11a73923-ce8b-424d-97a6-7deb14f917ae admin admin] The token cache is being invalidate because project 2fd2c804427749c68ff038d019a1628b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:49.986847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-11a73923-ce8b-424d-97a6-7deb14f917ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.987719 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-11a73923-ce8b-424d-97a6-7deb14f917ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:49.995466 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1017/2005] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:49 2022] DELETE /identity/v3/projects/2fd2c804427749c68ff038d019a1628b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:49.996917 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5470d91-71e5-4584-98d1-2144e59fccee None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:49.997270 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5470d91-71e5-4584-98d1-2144e59fccee None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:49.997616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5470d91-71e5-4584-98d1-2144e59fccee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.026850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a5470d91-71e5-4584-98d1-2144e59fccee None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:50.029326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a5470d91-71e5-4584-98d1-2144e59fccee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:50.049714 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:50.055839 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 989/2006] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:50.067653 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8c21e-6057-4bf8-a9bd-719e642c74b2 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:50.069183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b2d8c21e-6057-4bf8-a9bd-719e642c74b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:50.112532 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1018/2007] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:50.121401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538187a-a8bf-453d-8b81-be7728bb32fd None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.121695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538187a-a8bf-453d-8b81-be7728bb32fd None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.122089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538187a-a8bf-453d-8b81-be7728bb32fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.128177 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8c21e-6057-4bf8-a9bd-719e642c74b2 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.131825 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa110b-194b-459c-ab4e-0d7562b6cc6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.132440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa110b-194b-459c-ab4e-0d7562b6cc6a service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.132718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa110b-194b-459c-ab4e-0d7562b6cc6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.132991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa110b-194b-459c-ab4e-0d7562b6cc6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.142248 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa110b-194b-459c-ab4e-0d7562b6cc6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:50.142573 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5538187a-a8bf-453d-8b81-be7728bb32fd None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:50.143701 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa110b-194b-459c-ab4e-0d7562b6cc6a service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:50.143809 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5538187a-a8bf-453d-8b81-be7728bb32fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:50.151917 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1019/2008] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:50.153119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 990/2009] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:50.156471 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538187a-a8bf-453d-8b81-be7728bb32fd None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:50.165898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538187a-a8bf-453d-8b81-be7728bb32fd None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.170563 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7277b45-4e08-4161-9915-a73b669552a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.171154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7277b45-4e08-4161-9915-a73b669552a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.172154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7277b45-4e08-4161-9915-a73b669552a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.172527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7277b45-4e08-4161-9915-a73b669552a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.173177 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7277b45-4e08-4161-9915-a73b669552a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:50.173480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7277b45-4e08-4161-9915-a73b669552a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:50.175080 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7277b45-4e08-4161-9915-a73b669552a3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:50.190063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1020/2010] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:50.191993 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa110b-194b-459c-ab4e-0d7562b6cc6a service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:50.202550 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa110b-194b-459c-ab4e-0d7562b6cc6a service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.208232 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c95de-4e3b-4044-8f04-fdafc9ed86ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.209170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c95de-4e3b-4044-8f04-fdafc9ed86ba admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.209563 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c95de-4e3b-4044-8f04-fdafc9ed86ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.210025 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c95de-4e3b-4044-8f04-fdafc9ed86ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.211374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c95de-4e3b-4044-8f04-fdafc9ed86ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:50.212630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c95de-4e3b-4044-8f04-fdafc9ed86ba admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:50.246384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 991/2011] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:50 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:50.247548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7277b45-4e08-4161-9915-a73b669552a3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:50.258712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7277b45-4e08-4161-9915-a73b669552a3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.316239 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a67b8-8fad-4335-916e-5fc693790559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.316972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a67b8-8fad-4335-916e-5fc693790559 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.317259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a67b8-8fad-4335-916e-5fc693790559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.319955 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294a67b8-8fad-4335-916e-5fc693790559 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.320637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a67b8-8fad-4335-916e-5fc693790559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:50.322570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294a67b8-8fad-4335-916e-5fc693790559 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:50.347070 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1021/2012] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:50 2022] POST /identity/v3/users => generated 302 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:50.351060 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c95de-4e3b-4044-8f04-fdafc9ed86ba admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:50.362234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c95de-4e3b-4044-8f04-fdafc9ed86ba admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.366855 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91dd18-07b8-4fdf-ac6a-8f1539b7d79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.367396 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91dd18-07b8-4fdf-ac6a-8f1539b7d79f admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.367715 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91dd18-07b8-4fdf-ac6a-8f1539b7d79f admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.367993 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea91dd18-07b8-4fdf-ac6a-8f1539b7d79f admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.368634 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91dd18-07b8-4fdf-ac6a-8f1539b7d79f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:50.370228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea91dd18-07b8-4fdf-ac6a-8f1539b7d79f admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:50.383405 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 992/2013] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:50.386151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a67b8-8fad-4335-916e-5fc693790559 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:50.396748 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294a67b8-8fad-4335-916e-5fc693790559 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.401510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fdf55-60e3-4492-b41b-5b728058dcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.402124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fdf55-60e3-4492-b41b-5b728058dcbb admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.402413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fdf55-60e3-4492-b41b-5b728058dcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.402691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fdf55-60e3-4492-b41b-5b728058dcbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.403338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7fdf55-60e3-4492-b41b-5b728058dcbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:50.404990 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7fdf55-60e3-4492-b41b-5b728058dcbb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:50.416890 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1022/2014] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:50.421211 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91dd18-07b8-4fdf-ac6a-8f1539b7d79f admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:50.431532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea91dd18-07b8-4fdf-ac6a-8f1539b7d79f admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.436471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efd495-042b-4072-b449-de2ce49ddfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.437165 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efd495-042b-4072-b449-de2ce49ddfda admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.437448 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efd495-042b-4072-b449-de2ce49ddfda admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.437743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efd495-042b-4072-b449-de2ce49ddfda admin admin] PATH_INFO: `/v3/projects/3a4a55fb012c4a808a368a54a5942bc9/users/6659c567a9dd4da4b8a35f8b9259fc69/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.454071 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1efd495-042b-4072-b449-de2ce49ddfda admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4a55fb012c4a808a368a54a5942bc9, user_id=6659c567a9dd4da4b8a35f8b9259fc69, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:50.456836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1efd495-042b-4072-b449-de2ce49ddfda admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:50.470525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 993/2015] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:50 2022] PUT /identity/v3/projects/3a4a55fb012c4a808a368a54a5942bc9/users/6659c567a9dd4da4b8a35f8b9259fc69/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:50.476216 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82308020-48f1-4328-a67b-e1b24f00a0d6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.476506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82308020-48f1-4328-a67b-e1b24f00a0d6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.476780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82308020-48f1-4328-a67b-e1b24f00a0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.497414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-82308020-48f1-4328-a67b-e1b24f00a0d6 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:50.498661 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-82308020-48f1-4328-a67b-e1b24f00a0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:50.562062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:50.573527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1023/2016] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:50.589058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efd495-042b-4072-b449-de2ce49ddfda admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:50.589058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c1efd495-042b-4072-b449-de2ce49ddfda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:50.649008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efd495-042b-4072-b449-de2ce49ddfda admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.653037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0bb830-2ba0-46f4-8f1a-5f71662488ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.653645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bb830-2ba0-46f4-8f1a-5f71662488ea service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.653933 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bb830-2ba0-46f4-8f1a-5f71662488ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.654207 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0bb830-2ba0-46f4-8f1a-5f71662488ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.663824 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0bb830-2ba0-46f4-8f1a-5f71662488ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:50.665257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0bb830-2ba0-46f4-8f1a-5f71662488ea service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:50.675218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 994/2017] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:50.704015 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82308020-48f1-4328-a67b-e1b24f00a0d6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:50.705682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-82308020-48f1-4328-a67b-e1b24f00a0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:50.778219 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82308020-48f1-4328-a67b-e1b24f00a0d6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:50.782715 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8452904-d561-4cee-9d09-209db0e5d5ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:50.783348 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8452904-d561-4cee-9d09-209db0e5d5ac service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:50.783651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8452904-d561-4cee-9d09-209db0e5d5ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:50.783942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8452904-d561-4cee-9d09-209db0e5d5ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:50.785761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b8452904-d561-4cee-9d09-209db0e5d5ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:50.851323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8452904-d561-4cee-9d09-209db0e5d5ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:50.852820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8452904-d561-4cee-9d09-209db0e5d5ac service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:50.863454 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1024/2018] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:51.460974 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb533e-d3b7-49e7-9629-16ba8c9f780f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:51.461497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb533e-d3b7-49e7-9629-16ba8c9f780f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:51.461671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb533e-d3b7-49e7-9629-16ba8c9f780f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:51.492214 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-cfbb533e-d3b7-49e7-9629-16ba8c9f780f None None] MFA Rules not processed for user `6659c567a9dd4da4b8a35f8b9259fc69`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:51.494202 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-cfbb533e-d3b7-49e7-9629-16ba8c9f780f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:51.534622 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:51.576780 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 995/2019] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:51 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:51.582439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8452904-d561-4cee-9d09-209db0e5d5ac service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:51.598655 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8452904-d561-4cee-9d09-209db0e5d5ac service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:51.603690 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67a81b-6f7b-463a-9cfe-628456de6db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:51.604429 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67a81b-6f7b-463a-9cfe-628456de6db5 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:51.604774 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67a81b-6f7b-463a-9cfe-628456de6db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:51.605114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c67a81b-6f7b-463a-9cfe-628456de6db5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:51.605810 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67a81b-6f7b-463a-9cfe-628456de6db5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:51.607829 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c67a81b-6f7b-463a-9cfe-628456de6db5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:51.649283 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1025/2020] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:51 2022] POST /identity/v3/projects => generated 405 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:51.651242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb533e-d3b7-49e7-9629-16ba8c9f780f None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:51.666542 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb533e-d3b7-49e7-9629-16ba8c9f780f None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:51.736078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63465c-4ca5-41e0-b92c-1789df54a7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:51.736726 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63465c-4ca5-41e0-b92c-1789df54a7b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:51.737028 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63465c-4ca5-41e0-b92c-1789df54a7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:51.737306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63465c-4ca5-41e0-b92c-1789df54a7b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:51.737940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63465c-4ca5-41e0-b92c-1789df54a7b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:51.740051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63465c-4ca5-41e0-b92c-1789df54a7b1 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:51.762874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 996/2021] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:51 2022] POST /identity/v3/users => generated 309 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:51.766188 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67a81b-6f7b-463a-9cfe-628456de6db5 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:51.776501 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c67a81b-6f7b-463a-9cfe-628456de6db5 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:51.781222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a7612-d990-481e-a0f1-f3a16b91add4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:51.781925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a7612-d990-481e-a0f1-f3a16b91add4 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:51.782205 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a7612-d990-481e-a0f1-f3a16b91add4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:51.782476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9a7612-d990-481e-a0f1-f3a16b91add4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:51.783169 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a7612-d990-481e-a0f1-f3a16b91add4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:51.784835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9a7612-d990-481e-a0f1-f3a16b91add4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:51.795537 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1026/2022] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:51.799921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63465c-4ca5-41e0-b92c-1789df54a7b1 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:51.809955 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63465c-4ca5-41e0-b92c-1789df54a7b1 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:51.815381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35420c-4ef7-449c-be53-10b8637f7d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:51.816097 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35420c-4ef7-449c-be53-10b8637f7d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:51.816404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35420c-4ef7-449c-be53-10b8637f7d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:51.816705 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35420c-4ef7-449c-be53-10b8637f7d86 admin admin] PATH_INFO: `/v3/projects/bfbfbc0000ad49e1adfe779cf3872bf4/users/281db09f038f4b02b66e0cbdc69c377d/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:51.833558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35420c-4ef7-449c-be53-10b8637f7d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfbfbc0000ad49e1adfe779cf3872bf4, user_id=281db09f038f4b02b66e0cbdc69c377d, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:51.836350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35420c-4ef7-449c-be53-10b8637f7d86 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:51.850714 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 997/2023] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:51 2022] PUT /identity/v3/projects/bfbfbc0000ad49e1adfe779cf3872bf4/users/281db09f038f4b02b66e0cbdc69c377d/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:51.855057 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a7612-d990-481e-a0f1-f3a16b91add4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:51.865501 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9a7612-d990-481e-a0f1-f3a16b91add4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:51.929085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c1133-2462-410c-b33e-b493e85b4308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:51.929610 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c1133-2462-410c-b33e-b493e85b4308 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:51.929893 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c1133-2462-410c-b33e-b493e85b4308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:51.930168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c1133-2462-410c-b33e-b493e85b4308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:51.930808 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c1133-2462-410c-b33e-b493e85b4308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:51.932488 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c1133-2462-410c-b33e-b493e85b4308 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:51.943048 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1027/2024] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:51.947233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35420c-4ef7-449c-be53-10b8637f7d86 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:51.958276 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35420c-4ef7-449c-be53-10b8637f7d86 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:51.963044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70186a6a-9e6e-406e-ab7b-1d5946f2d8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:51.963764 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70186a6a-9e6e-406e-ab7b-1d5946f2d8bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:51.964048 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70186a6a-9e6e-406e-ab7b-1d5946f2d8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:51.964345 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70186a6a-9e6e-406e-ab7b-1d5946f2d8bc admin admin] PATH_INFO: `/v3/projects/bfbfbc0000ad49e1adfe779cf3872bf4/users/281db09f038f4b02b66e0cbdc69c377d/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:51.966253 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70186a6a-9e6e-406e-ab7b-1d5946f2d8bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfbfbc0000ad49e1adfe779cf3872bf4, user_id=281db09f038f4b02b66e0cbdc69c377d, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:51.969005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70186a6a-9e6e-406e-ab7b-1d5946f2d8bc admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:51.979159 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 998/2025] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:51 2022] PUT /identity/v3/projects/bfbfbc0000ad49e1adfe779cf3872bf4/users/281db09f038f4b02b66e0cbdc69c377d/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:52.665591 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a624d1-62f1-42b1-a061-d0446670c576 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:52.666101 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a624d1-62f1-42b1-a061-d0446670c576 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:52.666330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a624d1-62f1-42b1-a061-d0446670c576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:52.697956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-05a624d1-62f1-42b1-a061-d0446670c576 None None] MFA Rules not processed for user `281db09f038f4b02b66e0cbdc69c377d`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:52.699345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-05a624d1-62f1-42b1-a061-d0446670c576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:52.704895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:52.764797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70186a6a-9e6e-406e-ab7b-1d5946f2d8bc admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:52.767389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-70186a6a-9e6e-406e-ab7b-1d5946f2d8bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:52.809601 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1028/2026] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:52 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:52.864378 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70186a6a-9e6e-406e-ab7b-1d5946f2d8bc admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:52.871192 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedecb6d-294b-41a4-bf8c-1ce9ef447be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:52.872080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedecb6d-294b-41a4-bf8c-1ce9ef447be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:52.872591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedecb6d-294b-41a4-bf8c-1ce9ef447be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:52.872926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedecb6d-294b-41a4-bf8c-1ce9ef447be4 admin admin] PATH_INFO: `/v3/users/a9b8068041ed4006bd68b77e5c621390` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:52.880825 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedecb6d-294b-41a4-bf8c-1ce9ef447be4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9b8068041ed4006bd68b77e5c621390)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:52.888967 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedecb6d-294b-41a4-bf8c-1ce9ef447be4 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:52.926891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:52.941927 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-fedecb6d-294b-41a4-bf8c-1ce9ef447be4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9b8068041ed4006bd68b77e5c621390'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:52.942519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-fedecb6d-294b-41a4-bf8c-1ce9ef447be4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9b8068041ed4006bd68b77e5c621390'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:52.943128 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-fedecb6d-294b-41a4-bf8c-1ce9ef447be4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9b8068041ed4006bd68b77e5c621390'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:52.950361 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-fedecb6d-294b-41a4-bf8c-1ce9ef447be4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9b8068041ed4006bd68b77e5c621390'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:52.959762 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-fedecb6d-294b-41a4-bf8c-1ce9ef447be4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9b8068041ed4006bd68b77e5c621390'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:52.978472 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 999/2027] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:52 2022] DELETE /identity/v3/users/a9b8068041ed4006bd68b77e5c621390 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:52.982904 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67201713-a98e-41d2-918e-ff9601021119 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:52.983218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67201713-a98e-41d2-918e-ff9601021119 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:52.983502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67201713-a98e-41d2-918e-ff9601021119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:53.002917 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-67201713-a98e-41d2-918e-ff9601021119 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:53.004227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-67201713-a98e-41d2-918e-ff9601021119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:53.069701 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1029/2028] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:53.080481 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedecb6d-294b-41a4-bf8c-1ce9ef447be4 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:53.081926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fedecb6d-294b-41a4-bf8c-1ce9ef447be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:53.139435 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedecb6d-294b-41a4-bf8c-1ce9ef447be4 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:53.144098 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b29c5f-6f04-4d6c-b9e1-6df8b61b02af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:53.144713 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b29c5f-6f04-4d6c-b9e1-6df8b61b02af service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:53.144992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b29c5f-6f04-4d6c-b9e1-6df8b61b02af service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:53.145265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b29c5f-6f04-4d6c-b9e1-6df8b61b02af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:53.154379 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b29c5f-6f04-4d6c-b9e1-6df8b61b02af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:53.156102 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b29c5f-6f04-4d6c-b9e1-6df8b61b02af service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:53.165628 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1000/2029] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:53.539203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67201713-a98e-41d2-918e-ff9601021119 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:53.541234 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-67201713-a98e-41d2-918e-ff9601021119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:53.558363 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67201713-a98e-41d2-918e-ff9601021119 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:53.563946 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3245e0-85e5-402b-ad76-b9210ca26ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:53.564587 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3245e0-85e5-402b-ad76-b9210ca26ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:53.564867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3245e0-85e5-402b-ad76-b9210ca26ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:53.565143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3245e0-85e5-402b-ad76-b9210ca26ee8 admin admin] PATH_INFO: `/v3/projects/7dd5f1c8bfaa457085f5458126489a1a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:53.566284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3245e0-85e5-402b-ad76-b9210ca26ee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dd5f1c8bfaa457085f5458126489a1a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:53.568334 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3245e0-85e5-402b-ad76-b9210ca26ee8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:53.656665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ec3245e0-85e5-402b-ad76-b9210ca26ee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dd5f1c8bfaa457085f5458126489a1a', 'request_id': 'req-ec3245e0-85e5-402b-ad76-b9210ca26ee8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:53.673754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ec3245e0-85e5-402b-ad76-b9210ca26ee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dd5f1c8bfaa457085f5458126489a1a', 'request_id': 'req-ec3245e0-85e5-402b-ad76-b9210ca26ee8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:53.686523 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ec3245e0-85e5-402b-ad76-b9210ca26ee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dd5f1c8bfaa457085f5458126489a1a', 'request_id': 'req-ec3245e0-85e5-402b-ad76-b9210ca26ee8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:53.719888 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ec3245e0-85e5-402b-ad76-b9210ca26ee8 admin admin] The token cache is being invalidate because project 7dd5f1c8bfaa457085f5458126489a1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:53.720239 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ec3245e0-85e5-402b-ad76-b9210ca26ee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:53.721014 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ec3245e0-85e5-402b-ad76-b9210ca26ee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:53.723013 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1030/2030] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:53 2022] DELETE /identity/v3/projects/7dd5f1c8bfaa457085f5458126489a1a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:53.726925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b29c5f-6f04-4d6c-b9e1-6df8b61b02af service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:53.742462 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b29c5f-6f04-4d6c-b9e1-6df8b61b02af service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:53.825475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4ced8-8226-48de-b584-c492b15f3164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:53.826171 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4ced8-8226-48de-b584-c492b15f3164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:53.826501 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4ced8-8226-48de-b584-c492b15f3164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:53.826779 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4ced8-8226-48de-b584-c492b15f3164 admin admin] PATH_INFO: `/v3/users/f3b0c6781b0e4d1f92bc06ec80e43a49` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:53.828898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4ced8-8226-48de-b584-c492b15f3164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b0c6781b0e4d1f92bc06ec80e43a49)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:53.831428 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4ced8-8226-48de-b584-c492b15f3164 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:53.901635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-94e4ced8-8226-48de-b584-c492b15f3164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b0c6781b0e4d1f92bc06ec80e43a49'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:53.903844 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-94e4ced8-8226-48de-b584-c492b15f3164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b0c6781b0e4d1f92bc06ec80e43a49'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:53.904222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-94e4ced8-8226-48de-b584-c492b15f3164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b0c6781b0e4d1f92bc06ec80e43a49'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:53.910592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:53.914440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-94e4ced8-8226-48de-b584-c492b15f3164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b0c6781b0e4d1f92bc06ec80e43a49'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:53.933679 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-94e4ced8-8226-48de-b584-c492b15f3164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b0c6781b0e4d1f92bc06ec80e43a49'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:53.973261 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1001/2031] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:53 2022] DELETE /identity/v3/users/f3b0c6781b0e4d1f92bc06ec80e43a49 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:54.082447 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3245e0-85e5-402b-ad76-b9210ca26ee8 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:54.084397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ec3245e0-85e5-402b-ad76-b9210ca26ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:54.102555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3245e0-85e5-402b-ad76-b9210ca26ee8 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:54.123901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:54.164724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708606a-e2ea-48f6-98f0-3702f621dc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:54.165481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708606a-e2ea-48f6-98f0-3702f621dc99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:54.165830 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708606a-e2ea-48f6-98f0-3702f621dc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:54.166172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708606a-e2ea-48f6-98f0-3702f621dc99 admin admin] PATH_INFO: `/v3/users/6659c567a9dd4da4b8a35f8b9259fc69` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:54.167375 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708606a-e2ea-48f6-98f0-3702f621dc99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6659c567a9dd4da4b8a35f8b9259fc69)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:54.169410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708606a-e2ea-48f6-98f0-3702f621dc99 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:54.221889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8708606a-e2ea-48f6-98f0-3702f621dc99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6659c567a9dd4da4b8a35f8b9259fc69'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.222550 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8708606a-e2ea-48f6-98f0-3702f621dc99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6659c567a9dd4da4b8a35f8b9259fc69'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.229756 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8708606a-e2ea-48f6-98f0-3702f621dc99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6659c567a9dd4da4b8a35f8b9259fc69'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.238583 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8708606a-e2ea-48f6-98f0-3702f621dc99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6659c567a9dd4da4b8a35f8b9259fc69'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.252346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8708606a-e2ea-48f6-98f0-3702f621dc99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6659c567a9dd4da4b8a35f8b9259fc69'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.259263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1031/2032] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:54 2022] DELETE /identity/v3/users/6659c567a9dd4da4b8a35f8b9259fc69 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:54.318587 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4ced8-8226-48de-b584-c492b15f3164 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:54.320192 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-94e4ced8-8226-48de-b584-c492b15f3164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:54.354559 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4ced8-8226-48de-b584-c492b15f3164 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:54.415899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12009dc-d443-4d78-901a-29833d6af609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:54.416556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12009dc-d443-4d78-901a-29833d6af609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:54.416844 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12009dc-d443-4d78-901a-29833d6af609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:54.417120 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12009dc-d443-4d78-901a-29833d6af609 admin admin] PATH_INFO: `/v3/projects/6c5bbf74b53b4a4eabf15aff8a61f0ac` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:54.418520 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12009dc-d443-4d78-901a-29833d6af609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c5bbf74b53b4a4eabf15aff8a61f0ac)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:54.421077 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12009dc-d443-4d78-901a-29833d6af609 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:54.444921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708606a-e2ea-48f6-98f0-3702f621dc99 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:54.446382 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8708606a-e2ea-48f6-98f0-3702f621dc99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:54.461053 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708606a-e2ea-48f6-98f0-3702f621dc99 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:54.478479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c12009dc-d443-4d78-901a-29833d6af609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c5bbf74b53b4a4eabf15aff8a61f0ac', 'request_id': 'req-c12009dc-d443-4d78-901a-29833d6af609'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.484738 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c12009dc-d443-4d78-901a-29833d6af609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c5bbf74b53b4a4eabf15aff8a61f0ac', 'request_id': 'req-c12009dc-d443-4d78-901a-29833d6af609'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.493357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c12009dc-d443-4d78-901a-29833d6af609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c5bbf74b53b4a4eabf15aff8a61f0ac', 'request_id': 'req-c12009dc-d443-4d78-901a-29833d6af609'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.505287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c12009dc-d443-4d78-901a-29833d6af609 admin admin] The token cache is being invalidate because project 6c5bbf74b53b4a4eabf15aff8a61f0ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:54.505965 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c12009dc-d443-4d78-901a-29833d6af609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.506629 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c12009dc-d443-4d78-901a-29833d6af609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.508176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1002/2033] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:54 2022] DELETE /identity/v3/projects/6c5bbf74b53b4a4eabf15aff8a61f0ac => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:54.511881 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12009dc-d443-4d78-901a-29833d6af609 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:54.513501 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c12009dc-d443-4d78-901a-29833d6af609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:54.522389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd24b8-6d72-4935-acb1-b61808ae1493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:54.523026 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd24b8-6d72-4935-acb1-b61808ae1493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:54.523312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd24b8-6d72-4935-acb1-b61808ae1493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:54.523585 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd24b8-6d72-4935-acb1-b61808ae1493 admin admin] PATH_INFO: `/v3/projects/3a4a55fb012c4a808a368a54a5942bc9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:54.524760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd24b8-6d72-4935-acb1-b61808ae1493 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a4a55fb012c4a808a368a54a5942bc9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:54.526670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd24b8-6d72-4935-acb1-b61808ae1493 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:54.572491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:54.576868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3cbd24b8-6d72-4935-acb1-b61808ae1493 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a4a55fb012c4a808a368a54a5942bc9', 'request_id': 'req-3cbd24b8-6d72-4935-acb1-b61808ae1493'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.581320 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12009dc-d443-4d78-901a-29833d6af609 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:54.585211 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:54.585354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3cbd24b8-6d72-4935-acb1-b61808ae1493 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a4a55fb012c4a808a368a54a5942bc9', 'request_id': 'req-3cbd24b8-6d72-4935-acb1-b61808ae1493'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.592679 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3cbd24b8-6d72-4935-acb1-b61808ae1493 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a4a55fb012c4a808a368a54a5942bc9', 'request_id': 'req-3cbd24b8-6d72-4935-acb1-b61808ae1493'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.604915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3cbd24b8-6d72-4935-acb1-b61808ae1493 admin admin] The token cache is being invalidate because project 3a4a55fb012c4a808a368a54a5942bc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:54.605194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3cbd24b8-6d72-4935-acb1-b61808ae1493 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.605844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3cbd24b8-6d72-4935-acb1-b61808ae1493 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.607434 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1032/2034] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:54 2022] DELETE /identity/v3/projects/3a4a55fb012c4a808a368a54a5942bc9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:54.611393 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd24b8-6d72-4935-acb1-b61808ae1493 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:54.612761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3cbd24b8-6d72-4935-acb1-b61808ae1493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:54.646881 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1190f5b-7a8e-4a26-b510-3da34a5aac8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:54.647523 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1190f5b-7a8e-4a26-b510-3da34a5aac8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:54.647825 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1190f5b-7a8e-4a26-b510-3da34a5aac8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:54.648103 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1190f5b-7a8e-4a26-b510-3da34a5aac8d admin admin] PATH_INFO: `/v3/users/e424cff2ad614be986215629ee39376f` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:54.649237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1190f5b-7a8e-4a26-b510-3da34a5aac8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e424cff2ad614be986215629ee39376f)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:54.652116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1190f5b-7a8e-4a26-b510-3da34a5aac8d admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:54.689092 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd24b8-6d72-4935-acb1-b61808ae1493 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:54.696069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7623695-d06c-4268-84b7-fb5b21789807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:54.696819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7623695-d06c-4268-84b7-fb5b21789807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:54.697194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7623695-d06c-4268-84b7-fb5b21789807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:54.697769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7623695-d06c-4268-84b7-fb5b21789807 admin admin] PATH_INFO: `/v3/users/281db09f038f4b02b66e0cbdc69c377d` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:54.699374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7623695-d06c-4268-84b7-fb5b21789807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=281db09f038f4b02b66e0cbdc69c377d)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:54.701467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7623695-d06c-4268-84b7-fb5b21789807 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:54.708159 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b1190f5b-7a8e-4a26-b510-3da34a5aac8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e424cff2ad614be986215629ee39376f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.708930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b1190f5b-7a8e-4a26-b510-3da34a5aac8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e424cff2ad614be986215629ee39376f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.709692 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b1190f5b-7a8e-4a26-b510-3da34a5aac8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e424cff2ad614be986215629ee39376f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.717544 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b1190f5b-7a8e-4a26-b510-3da34a5aac8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e424cff2ad614be986215629ee39376f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.727832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b1190f5b-7a8e-4a26-b510-3da34a5aac8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e424cff2ad614be986215629ee39376f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.752255 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1003/2035] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:54 2022] DELETE /identity/v3/users/e424cff2ad614be986215629ee39376f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:54.767957 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a7623695-d06c-4268-84b7-fb5b21789807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '281db09f038f4b02b66e0cbdc69c377d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.768781 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a7623695-d06c-4268-84b7-fb5b21789807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '281db09f038f4b02b66e0cbdc69c377d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.778963 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a7623695-d06c-4268-84b7-fb5b21789807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '281db09f038f4b02b66e0cbdc69c377d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.791570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a7623695-d06c-4268-84b7-fb5b21789807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '281db09f038f4b02b66e0cbdc69c377d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.816904 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a7623695-d06c-4268-84b7-fb5b21789807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '281db09f038f4b02b66e0cbdc69c377d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:54.824315 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1033/2036] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:54 2022] DELETE /identity/v3/users/281db09f038f4b02b66e0cbdc69c377d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:55.069046 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1190f5b-7a8e-4a26-b510-3da34a5aac8d admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:55.071082 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b1190f5b-7a8e-4a26-b510-3da34a5aac8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:55.168295 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1190f5b-7a8e-4a26-b510-3da34a5aac8d admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:55.177737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7623695-d06c-4268-84b7-fb5b21789807 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:55.179714 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a7623695-d06c-4268-84b7-fb5b21789807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:55.209471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:55.246086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dcbc98-8f41-4ff6-8bec-6aaab7684365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:55.246952 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcbc98-8f41-4ff6-8bec-6aaab7684365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.247552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcbc98-8f41-4ff6-8bec-6aaab7684365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.247868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcbc98-8f41-4ff6-8bec-6aaab7684365 admin admin] PATH_INFO: `/v3/projects/bfbfbc0000ad49e1adfe779cf3872bf4` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.249358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dcbc98-8f41-4ff6-8bec-6aaab7684365 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfbfbc0000ad49e1adfe779cf3872bf4)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:55.252029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dcbc98-8f41-4ff6-8bec-6aaab7684365 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:55.260500 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:55.277171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7623695-d06c-4268-84b7-fb5b21789807 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:55.288580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ba6f5-7e4b-4a87-b167-d0e340bb28c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:55.288580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ba6f5-7e4b-4a87-b167-d0e340bb28c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.288580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ba6f5-7e4b-4a87-b167-d0e340bb28c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.288580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ba6f5-7e4b-4a87-b167-d0e340bb28c9 admin admin] PATH_INFO: `/v3/projects/19b64cfc511241f7a7198e1df82ef84c` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.288882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ba6f5-7e4b-4a87-b167-d0e340bb28c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b64cfc511241f7a7198e1df82ef84c)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:55.288882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ba6f5-7e4b-4a87-b167-d0e340bb28c9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:55.307479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-16dcbc98-8f41-4ff6-8bec-6aaab7684365 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfbfbc0000ad49e1adfe779cf3872bf4', 'request_id': 'req-16dcbc98-8f41-4ff6-8bec-6aaab7684365'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.313637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-16dcbc98-8f41-4ff6-8bec-6aaab7684365 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfbfbc0000ad49e1adfe779cf3872bf4', 'request_id': 'req-16dcbc98-8f41-4ff6-8bec-6aaab7684365'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.320589 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-16dcbc98-8f41-4ff6-8bec-6aaab7684365 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfbfbc0000ad49e1adfe779cf3872bf4', 'request_id': 'req-16dcbc98-8f41-4ff6-8bec-6aaab7684365'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.333656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-16dcbc98-8f41-4ff6-8bec-6aaab7684365 admin admin] The token cache is being invalidate because project bfbfbc0000ad49e1adfe779cf3872bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:55.334418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-16dcbc98-8f41-4ff6-8bec-6aaab7684365 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.335101 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-16dcbc98-8f41-4ff6-8bec-6aaab7684365 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.337868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d86ba6f5-7e4b-4a87-b167-d0e340bb28c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b64cfc511241f7a7198e1df82ef84c', 'request_id': 'req-d86ba6f5-7e4b-4a87-b167-d0e340bb28c9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.338012 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1004/2037] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:55 2022] DELETE /identity/v3/projects/bfbfbc0000ad49e1adfe779cf3872bf4 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:55.338814 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dcbc98-8f41-4ff6-8bec-6aaab7684365 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:55.340135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-16dcbc98-8f41-4ff6-8bec-6aaab7684365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:55.348405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d86ba6f5-7e4b-4a87-b167-d0e340bb28c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b64cfc511241f7a7198e1df82ef84c', 'request_id': 'req-d86ba6f5-7e4b-4a87-b167-d0e340bb28c9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.355942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d86ba6f5-7e4b-4a87-b167-d0e340bb28c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b64cfc511241f7a7198e1df82ef84c', 'request_id': 'req-d86ba6f5-7e4b-4a87-b167-d0e340bb28c9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.368650 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d86ba6f5-7e4b-4a87-b167-d0e340bb28c9 admin admin] The token cache is being invalidate because project 19b64cfc511241f7a7198e1df82ef84c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:55.369019 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d86ba6f5-7e4b-4a87-b167-d0e340bb28c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.369661 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-d86ba6f5-7e4b-4a87-b167-d0e340bb28c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.371172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1034/2038] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:55 2022] DELETE /identity/v3/projects/19b64cfc511241f7a7198e1df82ef84c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:55.372376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1118e-283d-48a2-a17a-b5e389ab419a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.372655 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1118e-283d-48a2-a17a-b5e389ab419a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.372924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1118e-283d-48a2-a17a-b5e389ab419a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.392800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-57a1118e-283d-48a2-a17a-b5e389ab419a None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:55.393998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-57a1118e-283d-48a2-a17a-b5e389ab419a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:55.401117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dcbc98-8f41-4ff6-8bec-6aaab7684365 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:55.449784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f9838-56fc-4366-ace9-4c7b3f3aaa39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:55.450398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f9838-56fc-4366-ace9-4c7b3f3aaa39 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.450675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f9838-56fc-4366-ace9-4c7b3f3aaa39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.450946 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f9838-56fc-4366-ace9-4c7b3f3aaa39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.452997 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1035/2039] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:55.453115 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-cb6f9838-56fc-4366-ace9-4c7b3f3aaa39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:55.454060 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570003-a5b0-4485-83b1-dc9e327f1e6a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.454347 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570003-a5b0-4485-83b1-dc9e327f1e6a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.454620 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570003-a5b0-4485-83b1-dc9e327f1e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.474367 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b8570003-a5b0-4485-83b1-dc9e327f1e6a None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:55.475935 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b8570003-a5b0-4485-83b1-dc9e327f1e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:55.484138 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1036/2040] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:55.485524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edfa47-d07b-46cf-ae75-0d82c4caf9d5 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.486319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edfa47-d07b-46cf-ae75-0d82c4caf9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.486590 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edfa47-d07b-46cf-ae75-0d82c4caf9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.500063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:55.519097 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-73edfa47-d07b-46cf-ae75-0d82c4caf9d5 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:55.520512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-73edfa47-d07b-46cf-ae75-0d82c4caf9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:55.529593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1037/2041] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:55.535684 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73edfa47-d07b-46cf-ae75-0d82c4caf9d5 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:55.537989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f9838-56fc-4366-ace9-4c7b3f3aaa39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:55.539527 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f9838-56fc-4366-ace9-4c7b3f3aaa39 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:55.548961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73edfa47-d07b-46cf-ae75-0d82c4caf9d5 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:55.553487 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f81c2-3bd3-4174-981c-4919971c71b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:55.553919 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1005/2042] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:55.554099 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f81c2-3bd3-4174-981c-4919971c71b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.554387 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f81c2-3bd3-4174-981c-4919971c71b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.554660 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f81c2-3bd3-4174-981c-4919971c71b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.555098 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c4bea-7070-4d19-a202-c877957ac273 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.555311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f81c2-3bd3-4174-981c-4919971c71b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:55.555405 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c4bea-7070-4d19-a202-c877957ac273 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.555613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f81c2-3bd3-4174-981c-4919971c71b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:55.555743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c4bea-7070-4d19-a202-c877957ac273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.557432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f81c2-3bd3-4174-981c-4919971c71b7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:55.571173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1038/2043] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:33:55.575253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f81c2-3bd3-4174-981c-4919971c71b7 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:55.580340 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-997c4bea-7070-4d19-a202-c877957ac273 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:55.581685 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-997c4bea-7070-4d19-a202-c877957ac273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:55.586568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f81c2-3bd3-4174-981c-4919971c71b7 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:55.589721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1006/2044] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:55.591524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724db45-f873-443d-81ae-8bec96a7fbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:55.592309 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724db45-f873-443d-81ae-8bec96a7fbfd admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.592755 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724db45-f873-443d-81ae-8bec96a7fbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.593644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724db45-f873-443d-81ae-8bec96a7fbfd admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.594306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c4bea-7070-4d19-a202-c877957ac273 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:55.594423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724db45-f873-443d-81ae-8bec96a7fbfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:55.596398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724db45-f873-443d-81ae-8bec96a7fbfd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:55.604613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c4bea-7070-4d19-a202-c877957ac273 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:55.612314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11630ae6-e96c-44bc-927e-efba9a56e533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:55.613031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11630ae6-e96c-44bc-927e-efba9a56e533 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.613382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11630ae6-e96c-44bc-927e-efba9a56e533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.613733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11630ae6-e96c-44bc-927e-efba9a56e533 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.614428 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11630ae6-e96c-44bc-927e-efba9a56e533 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:55.614697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11630ae6-e96c-44bc-927e-efba9a56e533 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:55.616585 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11630ae6-e96c-44bc-927e-efba9a56e533 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:55.631859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1039/2045] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:55 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:55.632036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1007/2046] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:33:55.633231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724db45-f873-443d-81ae-8bec96a7fbfd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:55.635664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11630ae6-e96c-44bc-927e-efba9a56e533 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:55.647471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11630ae6-e96c-44bc-927e-efba9a56e533 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:55.683019 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:55.715138 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724db45-f873-443d-81ae-8bec96a7fbfd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:55.720312 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f23e8-9c68-4b3e-a503-a16ce1f37d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:55.720967 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f23e8-9c68-4b3e-a503-a16ce1f37d4a admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.721265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f23e8-9c68-4b3e-a503-a16ce1f37d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.721531 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0f23e8-9c68-4b3e-a503-a16ce1f37d4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.721985 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015adecd-9d1d-46ec-ae6f-9f46eba74a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:55.722164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0f23e8-9c68-4b3e-a503-a16ce1f37d4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:55.722847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015adecd-9d1d-46ec-ae6f-9f46eba74a4d admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.723199 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015adecd-9d1d-46ec-ae6f-9f46eba74a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.723658 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015adecd-9d1d-46ec-ae6f-9f46eba74a4d admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.724101 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0f23e8-9c68-4b3e-a503-a16ce1f37d4a admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:55.724411 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015adecd-9d1d-46ec-ae6f-9f46eba74a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:55.726868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015adecd-9d1d-46ec-ae6f-9f46eba74a4d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:55.757196 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1040/2047] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:55 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:55.757385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1008/2048] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:55 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:55.758276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015adecd-9d1d-46ec-ae6f-9f46eba74a4d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:55.759548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f23e8-9c68-4b3e-a503-a16ce1f37d4a admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:55.769197 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015adecd-9d1d-46ec-ae6f-9f46eba74a4d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:55.836564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d910c63-dc43-4d98-b073-7292ef759c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:55.837184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d910c63-dc43-4d98-b073-7292ef759c93 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.837467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d910c63-dc43-4d98-b073-7292ef759c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.837737 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d910c63-dc43-4d98-b073-7292ef759c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.837932 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0f23e8-9c68-4b3e-a503-a16ce1f37d4a admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:55.838389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d910c63-dc43-4d98-b073-7292ef759c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:55.840070 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d910c63-dc43-4d98-b073-7292ef759c93 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:55.846370 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e01032-8b89-4a0b-aa8f-6d6a5959bdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:55.847015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e01032-8b89-4a0b-aa8f-6d6a5959bdfa admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.847438 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e01032-8b89-4a0b-aa8f-6d6a5959bdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.847838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e01032-8b89-4a0b-aa8f-6d6a5959bdfa admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.848510 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e01032-8b89-4a0b-aa8f-6d6a5959bdfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:55.850460 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e01032-8b89-4a0b-aa8f-6d6a5959bdfa admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:55.851655 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1041/2049] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:55.852381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d910c63-dc43-4d98-b073-7292ef759c93 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:55.854194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6d910c63-dc43-4d98-b073-7292ef759c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:55.871009 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d910c63-dc43-4d98-b073-7292ef759c93 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:55.877187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872831fa-c27f-408a-9927-3aec8c500b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:55.877836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872831fa-c27f-408a-9927-3aec8c500b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.878113 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872831fa-c27f-408a-9927-3aec8c500b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.878394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872831fa-c27f-408a-9927-3aec8c500b18 admin admin] PATH_INFO: `/v3/users/b3674d7b639f41179d4ff30bea8b522b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.879526 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872831fa-c27f-408a-9927-3aec8c500b18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3674d7b639f41179d4ff30bea8b522b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:55.881514 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872831fa-c27f-408a-9927-3aec8c500b18 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:55.881652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1009/2050] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:55 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:55.882794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e01032-8b89-4a0b-aa8f-6d6a5959bdfa admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:55.897164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e01032-8b89-4a0b-aa8f-6d6a5959bdfa admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:55.907400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee44b1-6d1a-4df2-87c9-cf5af321d7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:55.908690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee44b1-6d1a-4df2-87c9-cf5af321d7ed admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.909044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee44b1-6d1a-4df2-87c9-cf5af321d7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.909886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee44b1-6d1a-4df2-87c9-cf5af321d7ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.910616 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee44b1-6d1a-4df2-87c9-cf5af321d7ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:55.912338 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee44b1-6d1a-4df2-87c9-cf5af321d7ed admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:55.924323 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1010/2051] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:55.925875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee44b1-6d1a-4df2-87c9-cf5af321d7ed admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:55.936498 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee44b1-6d1a-4df2-87c9-cf5af321d7ed admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:55.944359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bcbe1-1034-4c31-99bd-4f492d6d4fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:55.945050 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bcbe1-1034-4c31-99bd-4f492d6d4fad admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.945446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bcbe1-1034-4c31-99bd-4f492d6d4fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.946163 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bcbe1-1034-4c31-99bd-4f492d6d4fad admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.947071 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293bcbe1-1034-4c31-99bd-4f492d6d4fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:55.949203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293bcbe1-1034-4c31-99bd-4f492d6d4fad admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:55.951679 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-872831fa-c27f-408a-9927-3aec8c500b18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3674d7b639f41179d4ff30bea8b522b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.953461 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-872831fa-c27f-408a-9927-3aec8c500b18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3674d7b639f41179d4ff30bea8b522b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.961016 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-872831fa-c27f-408a-9927-3aec8c500b18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3674d7b639f41179d4ff30bea8b522b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.961164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1011/2052] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:55.961988 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bcbe1-1034-4c31-99bd-4f492d6d4fad admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:55.975121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bcbe1-1034-4c31-99bd-4f492d6d4fad admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:55.975905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-872831fa-c27f-408a-9927-3aec8c500b18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3674d7b639f41179d4ff30bea8b522b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:55.980515 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc066aa-e32f-4017-a16e-1b0f42b7a771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:55.981182 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc066aa-e32f-4017-a16e-1b0f42b7a771 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:55.982333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc066aa-e32f-4017-a16e-1b0f42b7a771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:55.982333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc066aa-e32f-4017-a16e-1b0f42b7a771 admin admin] PATH_INFO: `/v3/projects/11c86a473d634104bf160997c023fe74/users/955673fcb75e413ba7ba97b7e8a3407f/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:55.984499 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:55.990905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-872831fa-c27f-408a-9927-3aec8c500b18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3674d7b639f41179d4ff30bea8b522b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:56.000754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1042/2053] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:55 2022] DELETE /identity/v3/users/b3674d7b639f41179d4ff30bea8b522b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:56.001588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872831fa-c27f-408a-9927-3aec8c500b18 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:56.004796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-872831fa-c27f-408a-9927-3aec8c500b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:56.006651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc066aa-e32f-4017-a16e-1b0f42b7a771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c86a473d634104bf160997c023fe74, user_id=955673fcb75e413ba7ba97b7e8a3407f, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:56.009464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc066aa-e32f-4017-a16e-1b0f42b7a771 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:56.025425 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1012/2054] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:55 2022] PUT /identity/v3/projects/11c86a473d634104bf160997c023fe74/users/955673fcb75e413ba7ba97b7e8a3407f/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:56.027415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b266321-51f0-4a24-ad94-34ca027c4c1a None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.028257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b266321-51f0-4a24-ad94-34ca027c4c1a None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.028549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b266321-51f0-4a24-ad94-34ca027c4c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.051092 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-7b266321-51f0-4a24-ad94-34ca027c4c1a None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:56.052614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7b266321-51f0-4a24-ad94-34ca027c4c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:56.064577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:56.107731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872831fa-c27f-408a-9927-3aec8c500b18 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:56.159036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1013/2055] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:56.160235 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42516e-ddf0-495a-a78b-e6fdb472f5bb None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.160521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42516e-ddf0-495a-a78b-e6fdb472f5bb None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.160799 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42516e-ddf0-495a-a78b-e6fdb472f5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.194557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2153763-e0bb-4f52-8732-66adc443b87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:56.195963 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2153763-e0bb-4f52-8732-66adc443b87c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.196343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2153763-e0bb-4f52-8732-66adc443b87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.196716 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2153763-e0bb-4f52-8732-66adc443b87c admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.198133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2153763-e0bb-4f52-8732-66adc443b87c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:56.200389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2153763-e0bb-4f52-8732-66adc443b87c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:56.200647 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-fb42516e-ddf0-495a-a78b-e6fdb472f5bb None None] MFA Rules not processed for user `955673fcb75e413ba7ba97b7e8a3407f`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:56.203139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fb42516e-ddf0-495a-a78b-e6fdb472f5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:56.214267 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1043/2056] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:56.215221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2153763-e0bb-4f52-8732-66adc443b87c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:56.217791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a2153763-e0bb-4f52-8732-66adc443b87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:56.304357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2153763-e0bb-4f52-8732-66adc443b87c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:56.307371 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1014/2057] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:56.308214 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42516e-ddf0-495a-a78b-e6fdb472f5bb None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:56.312835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29f5cf-1a1e-4292-9bb1-b7be0abf8760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:56.313616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29f5cf-1a1e-4292-9bb1-b7be0abf8760 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.313903 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29f5cf-1a1e-4292-9bb1-b7be0abf8760 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.314398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29f5cf-1a1e-4292-9bb1-b7be0abf8760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.319588 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42516e-ddf0-495a-a78b-e6fdb472f5bb None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:56.324587 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3e50e-c501-4225-92df-dc166aea4a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:56.325157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29f5cf-1a1e-4292-9bb1-b7be0abf8760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:56.325372 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3e50e-c501-4225-92df-dc166aea4a1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.325727 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3e50e-c501-4225-92df-dc166aea4a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.326072 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3e50e-c501-4225-92df-dc166aea4a1e admin admin] PATH_INFO: `/v3/projects/2bdabfd06f064b05a6d8ec06b4fca329/users/2af58e3ca6394bfa8ca34007c4ece01a/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.326662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29f5cf-1a1e-4292-9bb1-b7be0abf8760 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:56.338246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1044/2058] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:56.342822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3e50e-c501-4225-92df-dc166aea4a1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bdabfd06f064b05a6d8ec06b4fca329, user_id=2af58e3ca6394bfa8ca34007c4ece01a, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:56.345594 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3e50e-c501-4225-92df-dc166aea4a1e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:56.349840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29f5cf-1a1e-4292-9bb1-b7be0abf8760 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:56.353709 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7f29f5cf-1a1e-4292-9bb1-b7be0abf8760 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:56.363178 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1015/2059] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:56 2022] PUT /identity/v3/projects/2bdabfd06f064b05a6d8ec06b4fca329/users/2af58e3ca6394bfa8ca34007c4ece01a/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:56.373723 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cb07e-2ab3-47a8-8c3c-8c4503d793a6 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.373995 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cb07e-2ab3-47a8-8c3c-8c4503d793a6 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.375110 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cb07e-2ab3-47a8-8c3c-8c4503d793a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.379990 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29f5cf-1a1e-4292-9bb1-b7be0abf8760 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:56.427439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-963cb07e-2ab3-47a8-8c3c-8c4503d793a6 None None] MFA Rules not processed for user `2af58e3ca6394bfa8ca34007c4ece01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:56.429392 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-963cb07e-2ab3-47a8-8c3c-8c4503d793a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:56.465685 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22dc64-a72a-45f0-b012-2e50726c13a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:56.466419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22dc64-a72a-45f0-b012-2e50726c13a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.466776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22dc64-a72a-45f0-b012-2e50726c13a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.467121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22dc64-a72a-45f0-b012-2e50726c13a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.467856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22dc64-a72a-45f0-b012-2e50726c13a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:56.469896 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22dc64-a72a-45f0-b012-2e50726c13a0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:56.487820 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:56.504686 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1045/2060] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:56 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:56.509228 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22dc64-a72a-45f0-b012-2e50726c13a0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:56.515017 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:56.527153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1016/2061] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:56.534612 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963cb07e-2ab3-47a8-8c3c-8c4503d793a6 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:56.536086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-963cb07e-2ab3-47a8-8c3c-8c4503d793a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:56.602233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22dc64-a72a-45f0-b012-2e50726c13a0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:56.602809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963cb07e-2ab3-47a8-8c3c-8c4503d793a6 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:56.607747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc784939-9128-4e34-9b60-5aa85c42b071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:56.608523 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc784939-9128-4e34-9b60-5aa85c42b071 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.609372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc784939-9128-4e34-9b60-5aa85c42b071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.609609 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc784939-9128-4e34-9b60-5aa85c42b071 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.610246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc784939-9128-4e34-9b60-5aa85c42b071 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:56.612221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc784939-9128-4e34-9b60-5aa85c42b071 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:56.636593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1046/2062] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:56 2022] POST /identity/v3/users => generated 296 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:56.640386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc784939-9128-4e34-9b60-5aa85c42b071 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:56.653381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc784939-9128-4e34-9b60-5aa85c42b071 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:56.658465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827e802-aad2-46a4-b86b-1bd90f13c043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:56.659196 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827e802-aad2-46a4-b86b-1bd90f13c043 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.659661 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827e802-aad2-46a4-b86b-1bd90f13c043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.660069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827e802-aad2-46a4-b86b-1bd90f13c043 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.660900 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827e802-aad2-46a4-b86b-1bd90f13c043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:56.662810 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827e802-aad2-46a4-b86b-1bd90f13c043 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:56.675474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b0348-0abb-4924-a4d4-e18073d70066 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:56.676128 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b0348-0abb-4924-a4d4-e18073d70066 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.676440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b0348-0abb-4924-a4d4-e18073d70066 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.676696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b0348-0abb-4924-a4d4-e18073d70066 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.677049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1047/2063] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:56.681258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827e802-aad2-46a4-b86b-1bd90f13c043 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:56.693526 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827e802-aad2-46a4-b86b-1bd90f13c043 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:56.702392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9cf5c5-6f52-41dd-963e-f1e63894dc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:56.702392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cf5c5-6f52-41dd-963e-f1e63894dc0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.702392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cf5c5-6f52-41dd-963e-f1e63894dc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.702392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9cf5c5-6f52-41dd-963e-f1e63894dc0f admin admin] PATH_INFO: `/v3/projects/ceb4c7fc27684188bf814286191c9aa9/users/3dfa7f500871407c8a66bb550ee49140/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.718007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9cf5c5-6f52-41dd-963e-f1e63894dc0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb4c7fc27684188bf814286191c9aa9, user_id=3dfa7f500871407c8a66bb550ee49140, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:56.721021 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9cf5c5-6f52-41dd-963e-f1e63894dc0f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:56.737071 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1048/2064] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:56 2022] PUT /identity/v3/projects/ceb4c7fc27684188bf814286191c9aa9/users/3dfa7f500871407c8a66bb550ee49140/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:56.740593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9cf5c5-6f52-41dd-963e-f1e63894dc0f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:56.742416 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5d9cf5c5-6f52-41dd-963e-f1e63894dc0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:56.745836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b0348-0abb-4924-a4d4-e18073d70066 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:56.747281 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b0348-0abb-4924-a4d4-e18073d70066 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:56.757424 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1017/2065] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:56.758124 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b0348-0abb-4924-a4d4-e18073d70066 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:56.771492 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b0348-0abb-4924-a4d4-e18073d70066 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:56.816836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9cf5c5-6f52-41dd-963e-f1e63894dc0f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:56.823914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:56.824820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c9c43-c904-4f68-801c-a0338abe42c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:56.825680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c9c43-c904-4f68-801c-a0338abe42c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.826158 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c9c43-c904-4f68-801c-a0338abe42c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.826508 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c9c43-c904-4f68-801c-a0338abe42c5 admin admin] PATH_INFO: `/v3/projects/476e02daa1254c478297487abe5bbaf1` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.829033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c9c43-c904-4f68-801c-a0338abe42c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=476e02daa1254c478297487abe5bbaf1)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:56.830986 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c9c43-c904-4f68-801c-a0338abe42c5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:56.841897 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1f143-bfb7-4c8b-bf7c-37798347ab30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:56.842592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1f143-bfb7-4c8b-bf7c-37798347ab30 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.842870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1f143-bfb7-4c8b-bf7c-37798347ab30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.843732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1f143-bfb7-4c8b-bf7c-37798347ab30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.844529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1f143-bfb7-4c8b-bf7c-37798347ab30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:56.846144 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1f143-bfb7-4c8b-bf7c-37798347ab30 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:56.856890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1018/2066] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:56.862032 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1f143-bfb7-4c8b-bf7c-37798347ab30 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:56.873902 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1f143-bfb7-4c8b-bf7c-37798347ab30 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:56.883866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c55c9c43-c904-4f68-801c-a0338abe42c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '476e02daa1254c478297487abe5bbaf1', 'request_id': 'req-c55c9c43-c904-4f68-801c-a0338abe42c5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:56.893557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c55c9c43-c904-4f68-801c-a0338abe42c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '476e02daa1254c478297487abe5bbaf1', 'request_id': 'req-c55c9c43-c904-4f68-801c-a0338abe42c5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:56.896490 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:56.903781 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c55c9c43-c904-4f68-801c-a0338abe42c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '476e02daa1254c478297487abe5bbaf1', 'request_id': 'req-c55c9c43-c904-4f68-801c-a0338abe42c5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:56.918378 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c55c9c43-c904-4f68-801c-a0338abe42c5 admin admin] The token cache is being invalidate because project 476e02daa1254c478297487abe5bbaf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:56.918691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c55c9c43-c904-4f68-801c-a0338abe42c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:56.919359 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c55c9c43-c904-4f68-801c-a0338abe42c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:56.920890 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1049/2067] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:56 2022] DELETE /identity/v3/projects/476e02daa1254c478297487abe5bbaf1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:56.926194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb3edb-2eb2-470d-a48b-84c0ab1f420d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.926681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb3edb-2eb2-470d-a48b-84c0ab1f420d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.927016 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb3edb-2eb2-470d-a48b-84c0ab1f420d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.941336 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5323d-8bb1-4152-a92c-935d714c5a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:56.942008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5323d-8bb1-4152-a92c-935d714c5a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.942680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5323d-8bb1-4152-a92c-935d714c5a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.942680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5323d-8bb1-4152-a92c-935d714c5a5b admin admin] PATH_INFO: `/v3/projects/ceb4c7fc27684188bf814286191c9aa9/users/3dfa7f500871407c8a66bb550ee49140/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.944532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5323d-8bb1-4152-a92c-935d714c5a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb4c7fc27684188bf814286191c9aa9, user_id=3dfa7f500871407c8a66bb550ee49140, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:56.947197 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5323d-8bb1-4152-a92c-935d714c5a5b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:56.948619 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-37bb3edb-2eb2-470d-a48b-84c0ab1f420d None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:56.949869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-37bb3edb-2eb2-470d-a48b-84c0ab1f420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:56.959214 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1019/2068] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:56 2022] PUT /identity/v3/projects/ceb4c7fc27684188bf814286191c9aa9/users/3dfa7f500871407c8a66bb550ee49140/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:56.965874 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1050/2069] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:56.969932 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda5802-c97f-473f-9e42-1287ed36b3ef None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.970292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda5802-c97f-473f-9e42-1287ed36b3ef None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.970815 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda5802-c97f-473f-9e42-1287ed36b3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.970976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf4d24-3213-4be0-8450-53ee998bcf22 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:56.971265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf4d24-3213-4be0-8450-53ee998bcf22 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:56.971539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf4d24-3213-4be0-8450-53ee998bcf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:56.995120 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-9fda5802-c97f-473f-9e42-1287ed36b3ef None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:56.996404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-9fda5802-c97f-473f-9e42-1287ed36b3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:57.002492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-86bf4d24-3213-4be0-8450-53ee998bcf22 None None] MFA Rules not processed for user `3dfa7f500871407c8a66bb550ee49140`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:57.003702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-86bf4d24-3213-4be0-8450-53ee998bcf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:57.091003 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1051/2070] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:57.091194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1020/2071] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:57.093982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf4d24-3213-4be0-8450-53ee998bcf22 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:57.100531 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda5802-c97f-473f-9e42-1287ed36b3ef None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:57.102546 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-9fda5802-c97f-473f-9e42-1287ed36b3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:57.105723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf4d24-3213-4be0-8450-53ee998bcf22 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:57.111145 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d46107-a971-43a5-a6c2-a6a79b92e8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:57.111974 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d46107-a971-43a5-a6c2-a6a79b92e8dc admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:57.112327 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d46107-a971-43a5-a6c2-a6a79b92e8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:57.112741 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d46107-a971-43a5-a6c2-a6a79b92e8dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:57.113647 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d46107-a971-43a5-a6c2-a6a79b92e8dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:33:57.113959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d46107-a971-43a5-a6c2-a6a79b92e8dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:57.115797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d46107-a971-43a5-a6c2-a6a79b92e8dc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:57.132877 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1052/2072] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:33:57.134038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d46107-a971-43a5-a6c2-a6a79b92e8dc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:57.149807 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d46107-a971-43a5-a6c2-a6a79b92e8dc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:57.155262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8fb58-1e62-41f4-bc17-58c34410ba22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:57.156060 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8fb58-1e62-41f4-bc17-58c34410ba22 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:57.156497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8fb58-1e62-41f4-bc17-58c34410ba22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:57.156854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8fb58-1e62-41f4-bc17-58c34410ba22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:57.157624 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8fb58-1e62-41f4-bc17-58c34410ba22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:57.159846 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8fb58-1e62-41f4-bc17-58c34410ba22 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:57.171841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda5802-c97f-473f-9e42-1287ed36b3ef None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:57.175848 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7f036-c590-4dc3-b81b-efc1c06df4d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:57.176639 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7f036-c590-4dc3-b81b-efc1c06df4d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:57.176987 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7f036-c590-4dc3-b81b-efc1c06df4d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:57.177356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7f036-c590-4dc3-b81b-efc1c06df4d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:57.190739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7f036-c590-4dc3-b81b-efc1c06df4d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:57.191811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1053/2073] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:33:57 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:33:57.192228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7f036-c590-4dc3-b81b-efc1c06df4d5 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:57.195290 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8fb58-1e62-41f4-bc17-58c34410ba22 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:57.203491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1021/2074] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:33:57.265288 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8fb58-1e62-41f4-bc17-58c34410ba22 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:57.274381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182a98a-fb05-4bbb-84b7-87e96cda2004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:57.274381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182a98a-fb05-4bbb-84b7-87e96cda2004 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:57.274381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182a98a-fb05-4bbb-84b7-87e96cda2004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:57.274381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182a98a-fb05-4bbb-84b7-87e96cda2004 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:57.274672 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182a98a-fb05-4bbb-84b7-87e96cda2004 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:57.274672 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182a98a-fb05-4bbb-84b7-87e96cda2004 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:57.281550 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7f036-c590-4dc3-b81b-efc1c06df4d5 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:57.291850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7f036-c590-4dc3-b81b-efc1c06df4d5 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:57.299561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1054/2075] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:33:57 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:33:57.303603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182a98a-fb05-4bbb-84b7-87e96cda2004 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:57.314310 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182a98a-fb05-4bbb-84b7-87e96cda2004 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:57.315516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:57.321511 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f3697-a5bf-454f-b2cd-05a578483476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:57.322107 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f3697-a5bf-454f-b2cd-05a578483476 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:57.322386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f3697-a5bf-454f-b2cd-05a578483476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:57.322662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f3697-a5bf-454f-b2cd-05a578483476 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:57.323305 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f3697-a5bf-454f-b2cd-05a578483476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:57.324957 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f3697-a5bf-454f-b2cd-05a578483476 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:57.336821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1055/2076] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:33:57.340637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f3697-a5bf-454f-b2cd-05a578483476 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:57.351565 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f3697-a5bf-454f-b2cd-05a578483476 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:57.356486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e112f8-835e-4016-bc3a-e97ed8ef07dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:57.357087 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e112f8-835e-4016-bc3a-e97ed8ef07dc admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:57.357368 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e112f8-835e-4016-bc3a-e97ed8ef07dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:57.357646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e112f8-835e-4016-bc3a-e97ed8ef07dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:57.358287 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e112f8-835e-4016-bc3a-e97ed8ef07dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:57.360522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e112f8-835e-4016-bc3a-e97ed8ef07dc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:57.364590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ae6ad-dde7-4907-a4ba-43bc7f6789d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:57.365229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ae6ad-dde7-4907-a4ba-43bc7f6789d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:57.365543 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ae6ad-dde7-4907-a4ba-43bc7f6789d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:57.365820 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ae6ad-dde7-4907-a4ba-43bc7f6789d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:57.366269 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:57.368108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d60ae6ad-dde7-4907-a4ba-43bc7f6789d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:57.373465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1056/2077] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:33:57.376886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e112f8-835e-4016-bc3a-e97ed8ef07dc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:57.389303 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e112f8-835e-4016-bc3a-e97ed8ef07dc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:57.394274 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72732bd6-89c7-466c-9aa6-6d328c55f12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:57.394931 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72732bd6-89c7-466c-9aa6-6d328c55f12b admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:57.395217 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72732bd6-89c7-466c-9aa6-6d328c55f12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:57.395488 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72732bd6-89c7-466c-9aa6-6d328c55f12b admin admin] PATH_INFO: `/v3/projects/f3ea43b6f58f4706bc9789cba4bdfbe0/users/d6f83535318b463b86d64fda10211bb4/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:57.418593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72732bd6-89c7-466c-9aa6-6d328c55f12b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3ea43b6f58f4706bc9789cba4bdfbe0, user_id=d6f83535318b463b86d64fda10211bb4, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:57.422391 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72732bd6-89c7-466c-9aa6-6d328c55f12b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:57.438922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1057/2078] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:33:57 2022] PUT /identity/v3/projects/f3ea43b6f58f4706bc9789cba4bdfbe0/users/d6f83535318b463b86d64fda10211bb4/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:33:57.441803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ae6ad-dde7-4907-a4ba-43bc7f6789d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:57.445406 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ae6ad-dde7-4907-a4ba-43bc7f6789d1 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:57.450874 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f30ca9-df4c-4424-8398-3b2a7ef82ceb None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:57.451053 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f30ca9-df4c-4424-8398-3b2a7ef82ceb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:57.451310 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f30ca9-df4c-4424-8398-3b2a7ef82ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:57.457470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1022/2079] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:57.496525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-07f30ca9-df4c-4424-8398-3b2a7ef82ceb None None] MFA Rules not processed for user `d6f83535318b463b86d64fda10211bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:57.498116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-07f30ca9-df4c-4424-8398-3b2a7ef82ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:57.595933 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1058/2080] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:57.621439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ae6ad-dde7-4907-a4ba-43bc7f6789d1 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:57.630728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ae6ad-dde7-4907-a4ba-43bc7f6789d1 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:57.683761 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f12255-fc71-4fea-a998-703c828bc1c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:57.684492 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f12255-fc71-4fea-a998-703c828bc1c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:57.684922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f12255-fc71-4fea-a998-703c828bc1c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:57.685330 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f12255-fc71-4fea-a998-703c828bc1c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:57.694173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f12255-fc71-4fea-a998-703c828bc1c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:57.696001 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f12255-fc71-4fea-a998-703c828bc1c6 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:57.705458 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1023/2081] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:57.932153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f30ca9-df4c-4424-8398-3b2a7ef82ceb None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:57.933904 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-07f30ca9-df4c-4424-8398-3b2a7ef82ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:58.010556 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f30ca9-df4c-4424-8398-3b2a7ef82ceb None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:58.015826 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c990350-b28c-46b0-b80b-709f5f52968c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:58.016559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c990350-b28c-46b0-b80b-709f5f52968c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:58.016905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c990350-b28c-46b0-b80b-709f5f52968c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:58.017241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c990350-b28c-46b0-b80b-709f5f52968c admin admin] PATH_INFO: `/v3/users/a6f48aa2e6934f28b67cc9d090a79c08` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:58.018527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c990350-b28c-46b0-b80b-709f5f52968c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f48aa2e6934f28b67cc9d090a79c08)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:58.020624 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c990350-b28c-46b0-b80b-709f5f52968c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:58.074553 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3c990350-b28c-46b0-b80b-709f5f52968c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f48aa2e6934f28b67cc9d090a79c08'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:58.075194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3c990350-b28c-46b0-b80b-709f5f52968c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f48aa2e6934f28b67cc9d090a79c08'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:58.083228 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3c990350-b28c-46b0-b80b-709f5f52968c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f48aa2e6934f28b67cc9d090a79c08'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:58.090383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3c990350-b28c-46b0-b80b-709f5f52968c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f48aa2e6934f28b67cc9d090a79c08'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:58.096303 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:58.106059 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-3c990350-b28c-46b0-b80b-709f5f52968c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f48aa2e6934f28b67cc9d090a79c08'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:58.112175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1059/2082] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:57 2022] DELETE /identity/v3/users/a6f48aa2e6934f28b67cc9d090a79c08 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:58.116804 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05275c-157e-4fea-887c-e9a4135f29b4 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:58.117095 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05275c-157e-4fea-887c-e9a4135f29b4 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:58.117373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05275c-157e-4fea-887c-e9a4135f29b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:58.136087 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-ae05275c-157e-4fea-887c-e9a4135f29b4 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:58.137319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-ae05275c-157e-4fea-887c-e9a4135f29b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:58.197989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1024/2083] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:33:58.208778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c990350-b28c-46b0-b80b-709f5f52968c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:58.210313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3c990350-b28c-46b0-b80b-709f5f52968c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:58.213586 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc8bf6-ff21-4d7a-90ab-406f9975de33 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:58.213888 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc8bf6-ff21-4d7a-90ab-406f9975de33 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:58.214155 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc8bf6-ff21-4d7a-90ab-406f9975de33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:58.234073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-0cdc8bf6-ff21-4d7a-90ab-406f9975de33 None None] MFA Rules not processed for user `2af58e3ca6394bfa8ca34007c4ece01a`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:33:58.235362 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-0cdc8bf6-ff21-4d7a-90ab-406f9975de33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:58.247868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:58.273435 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c990350-b28c-46b0-b80b-709f5f52968c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:58.277089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bdfa9-6cf3-45e2-91ee-49ce16c72bd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:58.277687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bdfa9-6cf3-45e2-91ee-49ce16c72bd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:58.277965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bdfa9-6cf3-45e2-91ee-49ce16c72bd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:58.278235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914bdfa9-6cf3-45e2-91ee-49ce16c72bd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:58.288096 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bdfa9-6cf3-45e2-91ee-49ce16c72bd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:58.289522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914bdfa9-6cf3-45e2-91ee-49ce16c72bd3 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:58.292599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1025/2084] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:33:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:33:58.299034 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1060/2085] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:33:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:33:58.783440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc8bf6-ff21-4d7a-90ab-406f9975de33 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:58.784917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-0cdc8bf6-ff21-4d7a-90ab-406f9975de33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:58.801568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc8bf6-ff21-4d7a-90ab-406f9975de33 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:58.810278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cb708-8a1a-48b7-b178-a96a2a03635b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:58.810939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cb708-8a1a-48b7-b178-a96a2a03635b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:58.811226 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cb708-8a1a-48b7-b178-a96a2a03635b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:58.811498 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cb708-8a1a-48b7-b178-a96a2a03635b admin admin] PATH_INFO: `/v3/projects/737554c4d002424a9291b04f56fdf8aa` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:58.813689 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20cb708-8a1a-48b7-b178-a96a2a03635b admin admin] RBAC: Authorizing `identity:delete_project(project_id=737554c4d002424a9291b04f56fdf8aa)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:58.816771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20cb708-8a1a-48b7-b178-a96a2a03635b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:58.897722 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a20cb708-8a1a-48b7-b178-a96a2a03635b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '737554c4d002424a9291b04f56fdf8aa', 'request_id': 'req-a20cb708-8a1a-48b7-b178-a96a2a03635b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:58.907220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a20cb708-8a1a-48b7-b178-a96a2a03635b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '737554c4d002424a9291b04f56fdf8aa', 'request_id': 'req-a20cb708-8a1a-48b7-b178-a96a2a03635b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:58.915474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a20cb708-8a1a-48b7-b178-a96a2a03635b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '737554c4d002424a9291b04f56fdf8aa', 'request_id': 'req-a20cb708-8a1a-48b7-b178-a96a2a03635b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:58.937595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a20cb708-8a1a-48b7-b178-a96a2a03635b admin admin] The token cache is being invalidate because project 737554c4d002424a9291b04f56fdf8aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:58.938064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a20cb708-8a1a-48b7-b178-a96a2a03635b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:58.938855 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a20cb708-8a1a-48b7-b178-a96a2a03635b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:58.940713 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1026/2086] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:58 2022] DELETE /identity/v3/projects/737554c4d002424a9291b04f56fdf8aa => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:58.944593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bdfa9-6cf3-45e2-91ee-49ce16c72bd3 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:58.957455 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914bdfa9-6cf3-45e2-91ee-49ce16c72bd3 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:59.059318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52ec8b-590c-4f22-a8ab-e74d8f4842bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:59.060162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52ec8b-590c-4f22-a8ab-e74d8f4842bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:59.060548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52ec8b-590c-4f22-a8ab-e74d8f4842bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:59.060924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b52ec8b-590c-4f22-a8ab-e74d8f4842bf admin admin] PATH_INFO: `/v3/users/938712f670b84a829d093a56f8b45bb9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:59.064074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52ec8b-590c-4f22-a8ab-e74d8f4842bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=938712f670b84a829d093a56f8b45bb9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:59.066136 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b52ec8b-590c-4f22-a8ab-e74d8f4842bf admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:59.160364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5b52ec8b-590c-4f22-a8ab-e74d8f4842bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '938712f670b84a829d093a56f8b45bb9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.161371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5b52ec8b-590c-4f22-a8ab-e74d8f4842bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '938712f670b84a829d093a56f8b45bb9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.170980 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5b52ec8b-590c-4f22-a8ab-e74d8f4842bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '938712f670b84a829d093a56f8b45bb9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.180167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5b52ec8b-590c-4f22-a8ab-e74d8f4842bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '938712f670b84a829d093a56f8b45bb9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.199191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-5b52ec8b-590c-4f22-a8ab-e74d8f4842bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '938712f670b84a829d093a56f8b45bb9'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.208834 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1061/2087] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:58 2022] DELETE /identity/v3/users/938712f670b84a829d093a56f8b45bb9 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:59.503692 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cb708-8a1a-48b7-b178-a96a2a03635b admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:59.506374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a20cb708-8a1a-48b7-b178-a96a2a03635b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:59.525045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cb708-8a1a-48b7-b178-a96a2a03635b admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:59.588891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6cdd4-1fab-4dab-be92-4b8733643d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:59.589526 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6cdd4-1fab-4dab-be92-4b8733643d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:59.589796 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6cdd4-1fab-4dab-be92-4b8733643d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:59.590075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b6cdd4-1fab-4dab-be92-4b8733643d42 admin admin] PATH_INFO: `/v3/projects/d13dfe16c971434193e467f46f1ecd10` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:59.591320 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6cdd4-1fab-4dab-be92-4b8733643d42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d13dfe16c971434193e467f46f1ecd10)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:59.593966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b6cdd4-1fab-4dab-be92-4b8733643d42 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:59.647720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b9b6cdd4-1fab-4dab-be92-4b8733643d42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd13dfe16c971434193e467f46f1ecd10', 'request_id': 'req-b9b6cdd4-1fab-4dab-be92-4b8733643d42'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.648810 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:59.656664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b9b6cdd4-1fab-4dab-be92-4b8733643d42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd13dfe16c971434193e467f46f1ecd10', 'request_id': 'req-b9b6cdd4-1fab-4dab-be92-4b8733643d42'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.665402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b9b6cdd4-1fab-4dab-be92-4b8733643d42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd13dfe16c971434193e467f46f1ecd10', 'request_id': 'req-b9b6cdd4-1fab-4dab-be92-4b8733643d42'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.677576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b9b6cdd4-1fab-4dab-be92-4b8733643d42 admin admin] The token cache is being invalidate because project d13dfe16c971434193e467f46f1ecd10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:33:59.677849 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b9b6cdd4-1fab-4dab-be92-4b8733643d42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.678510 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b9b6cdd4-1fab-4dab-be92-4b8733643d42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.680061 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1027/2088] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:33:59 2022] DELETE /identity/v3/projects/d13dfe16c971434193e467f46f1ecd10 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:33:59.683973 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52ec8b-590c-4f22-a8ab-e74d8f4842bf admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:33:59.685453 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5b52ec8b-590c-4f22-a8ab-e74d8f4842bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:33:59.698324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b52ec8b-590c-4f22-a8ab-e74d8f4842bf admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:33:59.701856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:33:59.759102 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f4559-f444-4114-92a3-a3985da3d1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:33:59.759768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f4559-f444-4114-92a3-a3985da3d1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:33:59.760049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f4559-f444-4114-92a3-a3985da3d1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:33:59.760325 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f4559-f444-4114-92a3-a3985da3d1e5 admin admin] PATH_INFO: `/v3/users/b98ef000d03847e1bced097aab816ed2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:33:59.761431 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f4559-f444-4114-92a3-a3985da3d1e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b98ef000d03847e1bced097aab816ed2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:33:59.763341 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f4559-f444-4114-92a3-a3985da3d1e5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:33:59.808918 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8a0f4559-f444-4114-92a3-a3985da3d1e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b98ef000d03847e1bced097aab816ed2'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.809569 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8a0f4559-f444-4114-92a3-a3985da3d1e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b98ef000d03847e1bced097aab816ed2'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.817392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8a0f4559-f444-4114-92a3-a3985da3d1e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b98ef000d03847e1bced097aab816ed2'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.824523 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8a0f4559-f444-4114-92a3-a3985da3d1e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b98ef000d03847e1bced097aab816ed2'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.837597 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8a0f4559-f444-4114-92a3-a3985da3d1e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b98ef000d03847e1bced097aab816ed2'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:33:59.843521 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1062/2089] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:33:59 2022] DELETE /identity/v3/users/b98ef000d03847e1bced097aab816ed2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:00.295146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6cdd4-1fab-4dab-be92-4b8733643d42 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:00.297605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b9b6cdd4-1fab-4dab-be92-4b8733643d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:00.313201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b6cdd4-1fab-4dab-be92-4b8733643d42 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:00.366473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f4559-f444-4114-92a3-a3985da3d1e5 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:00.368637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8a0f4559-f444-4114-92a3-a3985da3d1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:00.404307 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689bd6b-fc1e-4452-bfdf-fa7d7477a148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:00.404950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689bd6b-fc1e-4452-bfdf-fa7d7477a148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:00.405236 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689bd6b-fc1e-4452-bfdf-fa7d7477a148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:00.405543 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689bd6b-fc1e-4452-bfdf-fa7d7477a148 admin admin] PATH_INFO: `/v3/projects/712cf6cde5e848d181d939a0a9d99dab` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:00.406723 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689bd6b-fc1e-4452-bfdf-fa7d7477a148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=712cf6cde5e848d181d939a0a9d99dab)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:00.408723 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689bd6b-fc1e-4452-bfdf-fa7d7477a148 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:00.448785 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f4559-f444-4114-92a3-a3985da3d1e5 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:00.457321 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94376c8f-ee01-48e8-8539-382d81303459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:00.458104 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94376c8f-ee01-48e8-8539-382d81303459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:00.458394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94376c8f-ee01-48e8-8539-382d81303459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:00.458676 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94376c8f-ee01-48e8-8539-382d81303459 admin admin] PATH_INFO: `/v3/users/d6f83535318b463b86d64fda10211bb4` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:00.460978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94376c8f-ee01-48e8-8539-382d81303459 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6f83535318b463b86d64fda10211bb4)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:00.463816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94376c8f-ee01-48e8-8539-382d81303459 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:00.480374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-0689bd6b-fc1e-4452-bfdf-fa7d7477a148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '712cf6cde5e848d181d939a0a9d99dab', 'request_id': 'req-0689bd6b-fc1e-4452-bfdf-fa7d7477a148'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:00.497104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-0689bd6b-fc1e-4452-bfdf-fa7d7477a148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '712cf6cde5e848d181d939a0a9d99dab', 'request_id': 'req-0689bd6b-fc1e-4452-bfdf-fa7d7477a148'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:00.504655 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:00.512387 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-0689bd6b-fc1e-4452-bfdf-fa7d7477a148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '712cf6cde5e848d181d939a0a9d99dab', 'request_id': 'req-0689bd6b-fc1e-4452-bfdf-fa7d7477a148'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:00.525852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-0689bd6b-fc1e-4452-bfdf-fa7d7477a148 admin admin] The token cache is being invalidate because project 712cf6cde5e848d181d939a0a9d99dab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:00.526270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-0689bd6b-fc1e-4452-bfdf-fa7d7477a148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:00.527153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-0689bd6b-fc1e-4452-bfdf-fa7d7477a148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:00.528736 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1028/2090] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:00 2022] DELETE /identity/v3/projects/712cf6cde5e848d181d939a0a9d99dab => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:00.534656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147728e4-dcdf-4601-8b4c-b7f776d02d8d None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:00.534945 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147728e4-dcdf-4601-8b4c-b7f776d02d8d None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:00.535247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147728e4-dcdf-4601-8b4c-b7f776d02d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:00.544155 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-94376c8f-ee01-48e8-8539-382d81303459 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6f83535318b463b86d64fda10211bb4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:00.545499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-94376c8f-ee01-48e8-8539-382d81303459 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6f83535318b463b86d64fda10211bb4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:00.556753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-94376c8f-ee01-48e8-8539-382d81303459 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6f83535318b463b86d64fda10211bb4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:00.575710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-147728e4-dcdf-4601-8b4c-b7f776d02d8d None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:00.583419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-94376c8f-ee01-48e8-8539-382d81303459 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6f83535318b463b86d64fda10211bb4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:00.584078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-147728e4-dcdf-4601-8b4c-b7f776d02d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:00.599957 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-94376c8f-ee01-48e8-8539-382d81303459 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6f83535318b463b86d64fda10211bb4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:00.607418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1063/2091] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:00 2022] DELETE /identity/v3/users/d6f83535318b463b86d64fda10211bb4 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:00.613034 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5156d7-6f5f-4656-8a24-9c6e0f1d6526 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:00.613551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5156d7-6f5f-4656-8a24-9c6e0f1d6526 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:00.613832 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5156d7-6f5f-4656-8a24-9c6e0f1d6526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:00.637610 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3f5156d7-6f5f-4656-8a24-9c6e0f1d6526 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:00.639234 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3f5156d7-6f5f-4656-8a24-9c6e0f1d6526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:00.652939 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1029/2092] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:00.662044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270ba94-c563-4953-8c02-d7d9f0eb0b64 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:00.662344 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270ba94-c563-4953-8c02-d7d9f0eb0b64 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:00.662635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270ba94-c563-4953-8c02-d7d9f0eb0b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:00.688745 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-6270ba94-c563-4953-8c02-d7d9f0eb0b64 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:00.689506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-6270ba94-c563-4953-8c02-d7d9f0eb0b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:00.694821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:00.723285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1064/2093] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:00.733010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5156d7-6f5f-4656-8a24-9c6e0f1d6526 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:00.735273 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3f5156d7-6f5f-4656-8a24-9c6e0f1d6526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:00.783685 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1030/2094] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:00.786481 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270ba94-c563-4953-8c02-d7d9f0eb0b64 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:00.798196 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270ba94-c563-4953-8c02-d7d9f0eb0b64 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:00.802795 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3db122-35bc-4b6f-a5d1-6a3757d08ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:00.804248 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3db122-35bc-4b6f-a5d1-6a3757d08ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:00.804248 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3db122-35bc-4b6f-a5d1-6a3757d08ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:00.804248 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3db122-35bc-4b6f-a5d1-6a3757d08ea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:00.804620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3db122-35bc-4b6f-a5d1-6a3757d08ea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:00.804928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3db122-35bc-4b6f-a5d1-6a3757d08ea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:00.806542 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3db122-35bc-4b6f-a5d1-6a3757d08ea3 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:00.808723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5156d7-6f5f-4656-8a24-9c6e0f1d6526 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:00.813016 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097887f-48e4-4cf5-ac50-180ceccfa246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:00.813725 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097887f-48e4-4cf5-ac50-180ceccfa246 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:00.814123 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097887f-48e4-4cf5-ac50-180ceccfa246 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:00.814510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097887f-48e4-4cf5-ac50-180ceccfa246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:00.821663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1031/2095] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:34:00.824295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097887f-48e4-4cf5-ac50-180ceccfa246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:00.825745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097887f-48e4-4cf5-ac50-180ceccfa246 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:00.826089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3db122-35bc-4b6f-a5d1-6a3757d08ea3 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:00.836494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1065/2096] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:34:00.837485 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3db122-35bc-4b6f-a5d1-6a3757d08ea3 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:00.844512 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaee1c4-4dca-450b-b780-c6c8ef452448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:00.845127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaee1c4-4dca-450b-b780-c6c8ef452448 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:00.845409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaee1c4-4dca-450b-b780-c6c8ef452448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:00.845685 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaee1c4-4dca-450b-b780-c6c8ef452448 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:00.846316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaee1c4-4dca-450b-b780-c6c8ef452448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:00.848311 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaee1c4-4dca-450b-b780-c6c8ef452448 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:00.884429 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1032/2097] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:00 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:00.888364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097887f-48e4-4cf5-ac50-180ceccfa246 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:00.902814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097887f-48e4-4cf5-ac50-180ceccfa246 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:00.966784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:00.984111 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ee41b-7fc7-42f5-9051-32bb7fe80804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:00.984888 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ee41b-7fc7-42f5-9051-32bb7fe80804 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:00.985296 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ee41b-7fc7-42f5-9051-32bb7fe80804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:00.985948 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ee41b-7fc7-42f5-9051-32bb7fe80804 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:00.987102 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ee41b-7fc7-42f5-9051-32bb7fe80804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:00.989174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ee41b-7fc7-42f5-9051-32bb7fe80804 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.033633 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1066/2098] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:00 2022] POST /identity/v3/users => generated 290 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:01.037340 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaee1c4-4dca-450b-b780-c6c8ef452448 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.056454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaee1c4-4dca-450b-b780-c6c8ef452448 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.064089 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29673bb-e58e-4af1-81e9-8b3dd4f609e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.064953 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29673bb-e58e-4af1-81e9-8b3dd4f609e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.065342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29673bb-e58e-4af1-81e9-8b3dd4f609e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.065716 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29673bb-e58e-4af1-81e9-8b3dd4f609e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.066643 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29673bb-e58e-4af1-81e9-8b3dd4f609e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.068702 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29673bb-e58e-4af1-81e9-8b3dd4f609e5 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.081351 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1033/2099] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:01.085142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ee41b-7fc7-42f5-9051-32bb7fe80804 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.098905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ee41b-7fc7-42f5-9051-32bb7fe80804 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.104895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026b9116-28b8-41cb-b378-5052d17d28e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.105667 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b9116-28b8-41cb-b378-5052d17d28e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.106053 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b9116-28b8-41cb-b378-5052d17d28e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.106427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026b9116-28b8-41cb-b378-5052d17d28e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.107397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b9116-28b8-41cb-b378-5052d17d28e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.111952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026b9116-28b8-41cb-b378-5052d17d28e3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.127434 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1067/2100] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:01.129739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29673bb-e58e-4af1-81e9-8b3dd4f609e5 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.141278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29673bb-e58e-4af1-81e9-8b3dd4f609e5 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.158336 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1e4ab-2240-4214-97ee-0f20aeebf570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.159433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1e4ab-2240-4214-97ee-0f20aeebf570 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.159861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1e4ab-2240-4214-97ee-0f20aeebf570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.161037 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1e4ab-2240-4214-97ee-0f20aeebf570 admin admin] PATH_INFO: `/v3/projects/3afeeb0654784107bfd2d606e6e7ff48/users/68ec5b1ba1264b2088419f6160344799/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.179784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1e4ab-2240-4214-97ee-0f20aeebf570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3afeeb0654784107bfd2d606e6e7ff48, user_id=68ec5b1ba1264b2088419f6160344799, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.182643 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1e4ab-2240-4214-97ee-0f20aeebf570 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.222348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1034/2101] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:01 2022] PUT /identity/v3/projects/3afeeb0654784107bfd2d606e6e7ff48/users/68ec5b1ba1264b2088419f6160344799/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:01.224072 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee74333-d901-47b7-ae93-24160a9ff4b0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.224365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee74333-d901-47b7-ae93-24160a9ff4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.224637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee74333-d901-47b7-ae93-24160a9ff4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.264406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-bee74333-d901-47b7-ae93-24160a9ff4b0 None None] MFA Rules not processed for user `68ec5b1ba1264b2088419f6160344799`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:01.266325 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bee74333-d901-47b7-ae93-24160a9ff4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:01.344109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1068/2102] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:01.349114 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1e4ab-2240-4214-97ee-0f20aeebf570 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.422416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1e4ab-2240-4214-97ee-0f20aeebf570 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.436365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0c354-8815-4dd8-bbfb-3f8e10bed83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.436981 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0c354-8815-4dd8-bbfb-3f8e10bed83a admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.437259 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0c354-8815-4dd8-bbfb-3f8e10bed83a admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.437535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0c354-8815-4dd8-bbfb-3f8e10bed83a admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.438172 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0c354-8815-4dd8-bbfb-3f8e10bed83a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.440223 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d0c354-8815-4dd8-bbfb-3f8e10bed83a admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.484193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1035/2103] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:01 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:01.488036 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee74333-d901-47b7-ae93-24160a9ff4b0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.499839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee74333-d901-47b7-ae93-24160a9ff4b0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.566435 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef5030-b491-4ca6-b06d-342d7ef6cca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.567043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef5030-b491-4ca6-b06d-342d7ef6cca4 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.567645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef5030-b491-4ca6-b06d-342d7ef6cca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.567645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef5030-b491-4ca6-b06d-342d7ef6cca4 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.568650 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef5030-b491-4ca6-b06d-342d7ef6cca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.570773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef5030-b491-4ca6-b06d-342d7ef6cca4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.577842 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0c354-8815-4dd8-bbfb-3f8e10bed83a admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.581199 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-66d0c354-8815-4dd8-bbfb-3f8e10bed83a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:01.617512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1069/2104] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:01 2022] POST /identity/v3/users => generated 298 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:01.618561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef5030-b491-4ca6-b06d-342d7ef6cca4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.620520 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0c354-8815-4dd8-bbfb-3f8e10bed83a admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.625799 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a10a2-9a1c-4815-aa40-b86e537d6b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.626550 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a10a2-9a1c-4815-aa40-b86e537d6b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.626898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a10a2-9a1c-4815-aa40-b86e537d6b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.627242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a10a2-9a1c-4815-aa40-b86e537d6b27 admin admin] PATH_INFO: `/v3/projects/f3ea43b6f58f4706bc9789cba4bdfbe0` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.628860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a10a2-9a1c-4815-aa40-b86e537d6b27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3ea43b6f58f4706bc9789cba4bdfbe0)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.630935 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a10a2-9a1c-4815-aa40-b86e537d6b27 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.631968 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef5030-b491-4ca6-b06d-342d7ef6cca4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.638467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c71214-be36-4722-8cb3-aa4f42908d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.639206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c71214-be36-4722-8cb3-aa4f42908d0a admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.639552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c71214-be36-4722-8cb3-aa4f42908d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.640080 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c71214-be36-4722-8cb3-aa4f42908d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.640954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c71214-be36-4722-8cb3-aa4f42908d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.643026 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c71214-be36-4722-8cb3-aa4f42908d0a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.653835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1070/2105] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:01.658668 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c71214-be36-4722-8cb3-aa4f42908d0a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.658886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:01.671715 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c71214-be36-4722-8cb3-aa4f42908d0a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.676825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274c27b-33e9-45ec-8775-b22c5b29ba68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.677491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274c27b-33e9-45ec-8775-b22c5b29ba68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.677773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274c27b-33e9-45ec-8775-b22c5b29ba68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.678048 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274c27b-33e9-45ec-8775-b22c5b29ba68 admin admin] PATH_INFO: `/v3/projects/aba82d281d4b4d209c09a51eb1eeab06/users/961dbad0300e477f9dbfb6070e5a0ac7/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.692283 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f82a10a2-9a1c-4815-aa40-b86e537d6b27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3ea43b6f58f4706bc9789cba4bdfbe0', 'request_id': 'req-f82a10a2-9a1c-4815-aa40-b86e537d6b27'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:01.693041 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5274c27b-33e9-45ec-8775-b22c5b29ba68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba82d281d4b4d209c09a51eb1eeab06, user_id=961dbad0300e477f9dbfb6070e5a0ac7, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.695739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5274c27b-33e9-45ec-8775-b22c5b29ba68 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.698163 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f82a10a2-9a1c-4815-aa40-b86e537d6b27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3ea43b6f58f4706bc9789cba4bdfbe0', 'request_id': 'req-f82a10a2-9a1c-4815-aa40-b86e537d6b27'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:01.706424 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f82a10a2-9a1c-4815-aa40-b86e537d6b27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3ea43b6f58f4706bc9789cba4bdfbe0', 'request_id': 'req-f82a10a2-9a1c-4815-aa40-b86e537d6b27'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:01.711690 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1071/2106] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:01 2022] PUT /identity/v3/projects/aba82d281d4b4d209c09a51eb1eeab06/users/961dbad0300e477f9dbfb6070e5a0ac7/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:01.715429 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274c27b-33e9-45ec-8775-b22c5b29ba68 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.720203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f82a10a2-9a1c-4815-aa40-b86e537d6b27 admin admin] The token cache is being invalidate because project f3ea43b6f58f4706bc9789cba4bdfbe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:01.720646 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f82a10a2-9a1c-4815-aa40-b86e537d6b27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:01.721903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f82a10a2-9a1c-4815-aa40-b86e537d6b27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:01.724348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1036/2107] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:01 2022] DELETE /identity/v3/projects/f3ea43b6f58f4706bc9789cba4bdfbe0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:01.729178 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2eff2a-1157-40a2-a1eb-29f1a7a30a7c None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.729541 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2eff2a-1157-40a2-a1eb-29f1a7a30a7c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.729879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2eff2a-1157-40a2-a1eb-29f1a7a30a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.731074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:01.750831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-9d2eff2a-1157-40a2-a1eb-29f1a7a30a7c None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:01.752279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-9d2eff2a-1157-40a2-a1eb-29f1a7a30a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:01.791125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5274c27b-33e9-45ec-8775-b22c5b29ba68 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.797266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e785a-4be8-449d-8aad-a2d045e9d551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.797882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e785a-4be8-449d-8aad-a2d045e9d551 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.798170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e785a-4be8-449d-8aad-a2d045e9d551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.798436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e785a-4be8-449d-8aad-a2d045e9d551 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.799084 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e785a-4be8-449d-8aad-a2d045e9d551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.801157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e785a-4be8-449d-8aad-a2d045e9d551 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.812589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1072/2108] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:34:01.816824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e785a-4be8-449d-8aad-a2d045e9d551 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.817867 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1037/2109] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:01.824995 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8e3e785a-4be8-449d-8aad-a2d045e9d551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:01.825768 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1229470-3444-4de9-9dd9-886ffea26b74 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.826059 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1229470-3444-4de9-9dd9-886ffea26b74 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.826341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1229470-3444-4de9-9dd9-886ffea26b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.839037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e785a-4be8-449d-8aad-a2d045e9d551 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.844635 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526d4d9-be40-43eb-a9b9-8f71450f8184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.845325 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526d4d9-be40-43eb-a9b9-8f71450f8184 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.845602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526d4d9-be40-43eb-a9b9-8f71450f8184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.845946 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526d4d9-be40-43eb-a9b9-8f71450f8184 admin admin] PATH_INFO: `/v3/projects/aba82d281d4b4d209c09a51eb1eeab06/users/961dbad0300e477f9dbfb6070e5a0ac7/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.847223 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-f1229470-3444-4de9-9dd9-886ffea26b74 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:01.847901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526d4d9-be40-43eb-a9b9-8f71450f8184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aba82d281d4b4d209c09a51eb1eeab06, user_id=961dbad0300e477f9dbfb6070e5a0ac7, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.848460 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f1229470-3444-4de9-9dd9-886ffea26b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:01.850613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526d4d9-be40-43eb-a9b9-8f71450f8184 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.856426 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1038/2110] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:01.860959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1229470-3444-4de9-9dd9-886ffea26b74 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.863447 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1073/2111] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:01 2022] PUT /identity/v3/projects/aba82d281d4b4d209c09a51eb1eeab06/users/961dbad0300e477f9dbfb6070e5a0ac7/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:01.870333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1229470-3444-4de9-9dd9-886ffea26b74 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.875099 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a14fa-dc5b-4641-a000-1c316deee1f5 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.875738 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a14fa-dc5b-4641-a000-1c316deee1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.876206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a14fa-dc5b-4641-a000-1c316deee1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.905198 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-823a14fa-dc5b-4641-a000-1c316deee1f5 None None] MFA Rules not processed for user `961dbad0300e477f9dbfb6070e5a0ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:01.906991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-823a14fa-dc5b-4641-a000-1c316deee1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:01.932510 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb53d26-48d8-4e2b-b3f7-0e60f4eea79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.933116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb53d26-48d8-4e2b-b3f7-0e60f4eea79a admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.933532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb53d26-48d8-4e2b-b3f7-0e60f4eea79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.933814 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb53d26-48d8-4e2b-b3f7-0e60f4eea79a admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.934477 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb53d26-48d8-4e2b-b3f7-0e60f4eea79a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:01.934779 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb53d26-48d8-4e2b-b3f7-0e60f4eea79a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.936454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb53d26-48d8-4e2b-b3f7-0e60f4eea79a admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.947887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1039/2112] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:34:01.952972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb53d26-48d8-4e2b-b3f7-0e60f4eea79a admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.965894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb53d26-48d8-4e2b-b3f7-0e60f4eea79a admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:01.971197 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474137e-e925-46ad-9982-1c194b275f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:01.971839 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474137e-e925-46ad-9982-1c194b275f10 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:01.972128 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474137e-e925-46ad-9982-1c194b275f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:01.972409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474137e-e925-46ad-9982-1c194b275f10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:01.973035 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474137e-e925-46ad-9982-1c194b275f10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:01.974945 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474137e-e925-46ad-9982-1c194b275f10 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:01.985376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1074/2113] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:01 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:01.996738 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a14fa-dc5b-4641-a000-1c316deee1f5 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:01.998150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-823a14fa-dc5b-4641-a000-1c316deee1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:02.002261 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1040/2114] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:01 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:34:02.006284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474137e-e925-46ad-9982-1c194b275f10 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:02.020189 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:02.058801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a14fa-dc5b-4641-a000-1c316deee1f5 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.071900 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474137e-e925-46ad-9982-1c194b275f10 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.077300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebf95d-fdf3-479f-b82e-34fe3cf1d917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.078672 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebf95d-fdf3-479f-b82e-34fe3cf1d917 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.078960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebf95d-fdf3-479f-b82e-34fe3cf1d917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.079234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebf95d-fdf3-479f-b82e-34fe3cf1d917 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.079887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ebf95d-fdf3-479f-b82e-34fe3cf1d917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.081821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ebf95d-fdf3-479f-b82e-34fe3cf1d917 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:02.092373 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:02.109294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1041/2115] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:02 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:34:02.110453 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebf95d-fdf3-479f-b82e-34fe3cf1d917 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:02.111530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8764b5c-31ff-4b56-af71-2a81848550ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.112236 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8764b5c-31ff-4b56-af71-2a81848550ed service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.112521 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8764b5c-31ff-4b56-af71-2a81848550ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.112799 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8764b5c-31ff-4b56-af71-2a81848550ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.120034 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebf95d-fdf3-479f-b82e-34fe3cf1d917 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.125490 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf320e-98c9-4171-96c6-427c66de5424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.126070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf320e-98c9-4171-96c6-427c66de5424 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.126348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf320e-98c9-4171-96c6-427c66de5424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.126622 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf320e-98c9-4171-96c6-427c66de5424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.127259 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf320e-98c9-4171-96c6-427c66de5424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.128878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf320e-98c9-4171-96c6-427c66de5424 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:02.142253 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1042/2116] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:02.143557 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf320e-98c9-4171-96c6-427c66de5424 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:02.153237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf320e-98c9-4171-96c6-427c66de5424 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.158467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21da049-b4fa-4edc-9f15-76acf1c760b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.159047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21da049-b4fa-4edc-9f15-76acf1c760b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.159327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21da049-b4fa-4edc-9f15-76acf1c760b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.159604 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21da049-b4fa-4edc-9f15-76acf1c760b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.160257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21da049-b4fa-4edc-9f15-76acf1c760b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.161822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21da049-b4fa-4edc-9f15-76acf1c760b0 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:02.172946 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1043/2117] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:34:02.177091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21da049-b4fa-4edc-9f15-76acf1c760b0 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:02.178079 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8764b5c-31ff-4b56-af71-2a81848550ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.179560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8764b5c-31ff-4b56-af71-2a81848550ed service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:02.187425 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21da049-b4fa-4edc-9f15-76acf1c760b0 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.189855 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1075/2118] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:34:02.192492 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bef1f6-9929-4b4c-be08-3af29bce1e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.193146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bef1f6-9929-4b4c-be08-3af29bce1e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.193427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bef1f6-9929-4b4c-be08-3af29bce1e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.193702 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bef1f6-9929-4b4c-be08-3af29bce1e94 admin admin] PATH_INFO: `/v3/projects/f3c01cea5def48e39c3b4eefc407123e/users/0329ee737db345bfaaa007ffb4c2fdee/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.209850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bef1f6-9929-4b4c-be08-3af29bce1e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3c01cea5def48e39c3b4eefc407123e, user_id=0329ee737db345bfaaa007ffb4c2fdee, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.213171 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bef1f6-9929-4b4c-be08-3af29bce1e94 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:02.226146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1044/2119] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:02 2022] PUT /identity/v3/projects/f3c01cea5def48e39c3b4eefc407123e/users/0329ee737db345bfaaa007ffb4c2fdee/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:02.236294 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49901a-fdea-42fb-8a2b-ad6d7accc93e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.236592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49901a-fdea-42fb-8a2b-ad6d7accc93e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.237088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49901a-fdea-42fb-8a2b-ad6d7accc93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.277636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ff49901a-fdea-42fb-8a2b-ad6d7accc93e None None] MFA Rules not processed for user `0329ee737db345bfaaa007ffb4c2fdee`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:02.279186 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ff49901a-fdea-42fb-8a2b-ad6d7accc93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:02.297976 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bef1f6-9929-4b4c-be08-3af29bce1e94 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:02.349440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1076/2120] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:02.356299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bef1f6-9929-4b4c-be08-3af29bce1e94 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.360274 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49901a-fdea-42fb-8a2b-ad6d7accc93e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:02.360450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca61392-b4bf-4a5a-8fa8-8c943e6ad9fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.361054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca61392-b4bf-4a5a-8fa8-8c943e6ad9fc service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.361338 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca61392-b4bf-4a5a-8fa8-8c943e6ad9fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.361615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca61392-b4bf-4a5a-8fa8-8c943e6ad9fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.361886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ff49901a-fdea-42fb-8a2b-ad6d7accc93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:02.363671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7ca61392-b4bf-4a5a-8fa8-8c943e6ad9fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:02.374699 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49901a-fdea-42fb-8a2b-ad6d7accc93e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:02.378469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f072fa1-7796-4443-bd65-24088b0120ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:02.379069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f072fa1-7796-4443-bd65-24088b0120ef service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:02.379348 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f072fa1-7796-4443-bd65-24088b0120ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:02.379648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f072fa1-7796-4443-bd65-24088b0120ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:02.387973 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f072fa1-7796-4443-bd65-24088b0120ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.389441 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f072fa1-7796-4443-bd65-24088b0120ef service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:02.400361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1077/2121] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:34:02.425351 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca61392-b4bf-4a5a-8fa8-8c943e6ad9fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:02.426786 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca61392-b4bf-4a5a-8fa8-8c943e6ad9fc service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:02.435511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1045/2122] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:34:05.362089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f072fa1-7796-4443-bd65-24088b0120ef service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:05.427827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f072fa1-7796-4443-bd65-24088b0120ef service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:05.433156 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bfe269-38d5-4819-a107-570aed7dea13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:05.433823 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfe269-38d5-4819-a107-570aed7dea13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:05.434106 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfe269-38d5-4819-a107-570aed7dea13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:05.434382 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bfe269-38d5-4819-a107-570aed7dea13 admin admin] PATH_INFO: `/v3/users/68ec5b1ba1264b2088419f6160344799` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:05.435511 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bfe269-38d5-4819-a107-570aed7dea13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68ec5b1ba1264b2088419f6160344799)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:05.437588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bfe269-38d5-4819-a107-570aed7dea13 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:05.440135 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:05.527961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e9bfe269-38d5-4819-a107-570aed7dea13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68ec5b1ba1264b2088419f6160344799'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:05.527961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e9bfe269-38d5-4819-a107-570aed7dea13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68ec5b1ba1264b2088419f6160344799'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:05.527961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e9bfe269-38d5-4819-a107-570aed7dea13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68ec5b1ba1264b2088419f6160344799'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:05.533364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e9bfe269-38d5-4819-a107-570aed7dea13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68ec5b1ba1264b2088419f6160344799'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:05.546154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-e9bfe269-38d5-4819-a107-570aed7dea13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68ec5b1ba1264b2088419f6160344799'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:05.552411 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1078/2123] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:05 2022] DELETE /identity/v3/users/68ec5b1ba1264b2088419f6160344799 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:05.556840 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3a595-2455-4bca-b3da-176555388c39 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:05.557322 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3a595-2455-4bca-b3da-176555388c39 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:05.557404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3a595-2455-4bca-b3da-176555388c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:05.576527 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-c7c3a595-2455-4bca-b3da-176555388c39 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:05.578670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c7c3a595-2455-4bca-b3da-176555388c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:05.583810 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:05.649619 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1046/2124] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:05.661427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bfe269-38d5-4819-a107-570aed7dea13 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:05.662993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e9bfe269-38d5-4819-a107-570aed7dea13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:05.734624 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bfe269-38d5-4819-a107-570aed7dea13 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:05.738920 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94080d46-92c7-45c6-9ddd-a0b7155be01c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:05.739549 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94080d46-92c7-45c6-9ddd-a0b7155be01c service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:05.739865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94080d46-92c7-45c6-9ddd-a0b7155be01c service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:05.740147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94080d46-92c7-45c6-9ddd-a0b7155be01c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:05.760149 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94080d46-92c7-45c6-9ddd-a0b7155be01c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:05.761600 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94080d46-92c7-45c6-9ddd-a0b7155be01c service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:05.775052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1079/2125] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:06.117730 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3a595-2455-4bca-b3da-176555388c39 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:06.119240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c7c3a595-2455-4bca-b3da-176555388c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:06.132148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3a595-2455-4bca-b3da-176555388c39 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:06.137506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10190199-75b8-42df-8f48-be38ea77863e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:06.138270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10190199-75b8-42df-8f48-be38ea77863e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:06.138586 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10190199-75b8-42df-8f48-be38ea77863e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:06.138821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10190199-75b8-42df-8f48-be38ea77863e admin admin] PATH_INFO: `/v3/projects/3afeeb0654784107bfd2d606e6e7ff48` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:06.140198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10190199-75b8-42df-8f48-be38ea77863e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3afeeb0654784107bfd2d606e6e7ff48)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:06.142234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10190199-75b8-42df-8f48-be38ea77863e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:06.193970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-10190199-75b8-42df-8f48-be38ea77863e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3afeeb0654784107bfd2d606e6e7ff48', 'request_id': 'req-10190199-75b8-42df-8f48-be38ea77863e'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:06.199525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-10190199-75b8-42df-8f48-be38ea77863e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3afeeb0654784107bfd2d606e6e7ff48', 'request_id': 'req-10190199-75b8-42df-8f48-be38ea77863e'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:06.207120 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-10190199-75b8-42df-8f48-be38ea77863e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3afeeb0654784107bfd2d606e6e7ff48', 'request_id': 'req-10190199-75b8-42df-8f48-be38ea77863e'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:06.218884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-10190199-75b8-42df-8f48-be38ea77863e admin admin] The token cache is being invalidate because project 3afeeb0654784107bfd2d606e6e7ff48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:06.219171 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-10190199-75b8-42df-8f48-be38ea77863e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:06.219832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-10190199-75b8-42df-8f48-be38ea77863e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:06.221326 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1047/2126] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:06 2022] DELETE /identity/v3/projects/3afeeb0654784107bfd2d606e6e7ff48 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:06.225409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94080d46-92c7-45c6-9ddd-a0b7155be01c service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:06.235616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94080d46-92c7-45c6-9ddd-a0b7155be01c service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:06.299865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c3e5c-b2ae-466f-99fb-23eb9946bb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:06.300559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c3e5c-b2ae-466f-99fb-23eb9946bb81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:06.300783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c3e5c-b2ae-466f-99fb-23eb9946bb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:06.301055 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c3e5c-b2ae-466f-99fb-23eb9946bb81 admin admin] PATH_INFO: `/v3/users/961dbad0300e477f9dbfb6070e5a0ac7` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:06.302178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c3e5c-b2ae-466f-99fb-23eb9946bb81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=961dbad0300e477f9dbfb6070e5a0ac7)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:06.304210 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c3e5c-b2ae-466f-99fb-23eb9946bb81 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:06.331828 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:06.352284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-471c3e5c-b2ae-466f-99fb-23eb9946bb81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '961dbad0300e477f9dbfb6070e5a0ac7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:06.352945 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-471c3e5c-b2ae-466f-99fb-23eb9946bb81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '961dbad0300e477f9dbfb6070e5a0ac7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:06.361380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-471c3e5c-b2ae-466f-99fb-23eb9946bb81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '961dbad0300e477f9dbfb6070e5a0ac7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:06.369209 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-471c3e5c-b2ae-466f-99fb-23eb9946bb81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '961dbad0300e477f9dbfb6070e5a0ac7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:06.381696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-471c3e5c-b2ae-466f-99fb-23eb9946bb81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '961dbad0300e477f9dbfb6070e5a0ac7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:06.387936 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1080/2127] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:06 2022] DELETE /identity/v3/users/961dbad0300e477f9dbfb6070e5a0ac7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:06.673731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10190199-75b8-42df-8f48-be38ea77863e admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:06.675802 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-10190199-75b8-42df-8f48-be38ea77863e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:06.690445 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10190199-75b8-42df-8f48-be38ea77863e admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:06.758598 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146cdfc-411b-4b66-81e8-53343e7b58ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:06.759240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146cdfc-411b-4b66-81e8-53343e7b58ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:06.759522 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146cdfc-411b-4b66-81e8-53343e7b58ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:06.759824 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146cdfc-411b-4b66-81e8-53343e7b58ca admin admin] PATH_INFO: `/v3/projects/aba82d281d4b4d209c09a51eb1eeab06` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:06.760962 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146cdfc-411b-4b66-81e8-53343e7b58ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=aba82d281d4b4d209c09a51eb1eeab06)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:06.762872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146cdfc-411b-4b66-81e8-53343e7b58ca admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:06.800446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:06.809950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6146cdfc-411b-4b66-81e8-53343e7b58ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aba82d281d4b4d209c09a51eb1eeab06', 'request_id': 'req-6146cdfc-411b-4b66-81e8-53343e7b58ca'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:06.815602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6146cdfc-411b-4b66-81e8-53343e7b58ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aba82d281d4b4d209c09a51eb1eeab06', 'request_id': 'req-6146cdfc-411b-4b66-81e8-53343e7b58ca'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:06.823088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6146cdfc-411b-4b66-81e8-53343e7b58ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aba82d281d4b4d209c09a51eb1eeab06', 'request_id': 'req-6146cdfc-411b-4b66-81e8-53343e7b58ca'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:06.834614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6146cdfc-411b-4b66-81e8-53343e7b58ca admin admin] The token cache is being invalidate because project aba82d281d4b4d209c09a51eb1eeab06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:06.834905 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6146cdfc-411b-4b66-81e8-53343e7b58ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:06.835540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6146cdfc-411b-4b66-81e8-53343e7b58ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:06.837055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1048/2128] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:06 2022] DELETE /identity/v3/projects/aba82d281d4b4d209c09a51eb1eeab06 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:06.842060 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c019a2-4776-471a-b6fc-e82f926c169b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:06.842294 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c019a2-4776-471a-b6fc-e82f926c169b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:06.842579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c019a2-4776-471a-b6fc-e82f926c169b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:06.863116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-34c019a2-4776-471a-b6fc-e82f926c169b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:06.864354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-34c019a2-4776-471a-b6fc-e82f926c169b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:06.927210 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1081/2129] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:06.935014 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2251d71-0441-488f-893d-41e4b21f48cd None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:06.935373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2251d71-0441-488f-893d-41e4b21f48cd None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:06.935679 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2251d71-0441-488f-893d-41e4b21f48cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:06.956641 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-d2251d71-0441-488f-893d-41e4b21f48cd None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:06.957854 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d2251d71-0441-488f-893d-41e4b21f48cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:06.965843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1049/2130] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:06.971909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c019a2-4776-471a-b6fc-e82f926c169b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:06.982703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c019a2-4776-471a-b6fc-e82f926c169b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:06.987193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff32e3-e74c-4e44-a1d9-c1666392598b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:06.987923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff32e3-e74c-4e44-a1d9-c1666392598b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:06.988204 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff32e3-e74c-4e44-a1d9-c1666392598b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:06.988483 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff32e3-e74c-4e44-a1d9-c1666392598b admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:06.989142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff32e3-e74c-4e44-a1d9-c1666392598b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:06.989449 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff32e3-e74c-4e44-a1d9-c1666392598b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:06.991075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ff32e3-e74c-4e44-a1d9-c1666392598b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:07.007195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1082/2131] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:34:07.011547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2251d71-0441-488f-893d-41e4b21f48cd None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:07.027917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2251d71-0441-488f-893d-41e4b21f48cd None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:07.027917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e151e-0f27-4ed1-a82e-3ba7c7c988e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:07.027917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e151e-0f27-4ed1-a82e-3ba7c7c988e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:07.028264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e151e-0f27-4ed1-a82e-3ba7c7c988e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:07.028264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e151e-0f27-4ed1-a82e-3ba7c7c988e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:07.029481 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e151e-0f27-4ed1-a82e-3ba7c7c988e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:07.032825 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e151e-0f27-4ed1-a82e-3ba7c7c988e8 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:07.067753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1050/2132] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:07 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:07.068977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff32e3-e74c-4e44-a1d9-c1666392598b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:07.082486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ff32e3-e74c-4e44-a1d9-c1666392598b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:07.150415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2507988-758d-410e-8fc1-8c90d751152c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:07.151046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2507988-758d-410e-8fc1-8c90d751152c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:07.151330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2507988-758d-410e-8fc1-8c90d751152c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:07.151599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2507988-758d-410e-8fc1-8c90d751152c admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:07.152266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2507988-758d-410e-8fc1-8c90d751152c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:07.154183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2507988-758d-410e-8fc1-8c90d751152c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:07.187896 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1083/2133] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:07 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:07.187896 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2507988-758d-410e-8fc1-8c90d751152c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:07.188359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e151e-0f27-4ed1-a82e-3ba7c7c988e8 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:07.189420 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-408e151e-0f27-4ed1-a82e-3ba7c7c988e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:07.193124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2507988-758d-410e-8fc1-8c90d751152c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:07.198195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3c4b6-e672-422f-9d92-44f2e9a19f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:07.198793 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3c4b6-e672-422f-9d92-44f2e9a19f14 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:07.199075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3c4b6-e672-422f-9d92-44f2e9a19f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:07.199346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3c4b6-e672-422f-9d92-44f2e9a19f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:07.200036 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3c4b6-e672-422f-9d92-44f2e9a19f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:07.201757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3c4b6-e672-422f-9d92-44f2e9a19f14 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:07.204171 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e151e-0f27-4ed1-a82e-3ba7c7c988e8 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:07.209892 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc147aaa-941c-4c6f-aa8b-d5a331bb1b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:07.210933 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc147aaa-941c-4c6f-aa8b-d5a331bb1b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:07.211284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc147aaa-941c-4c6f-aa8b-d5a331bb1b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:07.211720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc147aaa-941c-4c6f-aa8b-d5a331bb1b23 admin admin] PATH_INFO: `/v3/users/955673fcb75e413ba7ba97b7e8a3407f` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:07.212880 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc147aaa-941c-4c6f-aa8b-d5a331bb1b23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=955673fcb75e413ba7ba97b7e8a3407f)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:07.216785 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1084/2134] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:07.216913 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc147aaa-941c-4c6f-aa8b-d5a331bb1b23 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:07.217642 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3c4b6-e672-422f-9d92-44f2e9a19f14 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:07.228083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3c4b6-e672-422f-9d92-44f2e9a19f14 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:07.232744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064791d3-de0d-4183-8642-96de9b8cb59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:07.233368 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064791d3-de0d-4183-8642-96de9b8cb59a admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:07.233830 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064791d3-de0d-4183-8642-96de9b8cb59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:07.234104 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064791d3-de0d-4183-8642-96de9b8cb59a admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:07.234742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064791d3-de0d-4183-8642-96de9b8cb59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:07.236395 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064791d3-de0d-4183-8642-96de9b8cb59a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:07.238635 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:07.250210 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1085/2135] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:34:07.253576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064791d3-de0d-4183-8642-96de9b8cb59a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:07.266617 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064791d3-de0d-4183-8642-96de9b8cb59a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:07.278499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19734cf0-4ed5-4825-b5e1-aae722fc04c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:07.279225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19734cf0-4ed5-4825-b5e1-aae722fc04c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:07.279519 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19734cf0-4ed5-4825-b5e1-aae722fc04c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:07.279889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19734cf0-4ed5-4825-b5e1-aae722fc04c5 admin admin] PATH_INFO: `/v3/projects/e5949afd7d8b483db23f203952fc0547/users/c1d48bafec3045389599887b45ed9ee4/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:07.291875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-dc147aaa-941c-4c6f-aa8b-d5a331bb1b23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '955673fcb75e413ba7ba97b7e8a3407f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:07.292590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-dc147aaa-941c-4c6f-aa8b-d5a331bb1b23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '955673fcb75e413ba7ba97b7e8a3407f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:07.293545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-dc147aaa-941c-4c6f-aa8b-d5a331bb1b23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '955673fcb75e413ba7ba97b7e8a3407f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:07.298284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19734cf0-4ed5-4825-b5e1-aae722fc04c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5949afd7d8b483db23f203952fc0547, user_id=c1d48bafec3045389599887b45ed9ee4, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:07.301003 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19734cf0-4ed5-4825-b5e1-aae722fc04c5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:07.302796 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-dc147aaa-941c-4c6f-aa8b-d5a331bb1b23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '955673fcb75e413ba7ba97b7e8a3407f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:07.313614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-dc147aaa-941c-4c6f-aa8b-d5a331bb1b23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '955673fcb75e413ba7ba97b7e8a3407f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:07.319095 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1086/2136] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:07 2022] PUT /identity/v3/projects/e5949afd7d8b483db23f203952fc0547/users/c1d48bafec3045389599887b45ed9ee4/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:07.330730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a0aa3-8b7e-495d-b94a-e855a3c08507 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:07.331012 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a0aa3-8b7e-495d-b94a-e855a3c08507 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:07.331323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a0aa3-8b7e-495d-b94a-e855a3c08507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:07.333876 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1051/2137] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:07 2022] DELETE /identity/v3/users/955673fcb75e413ba7ba97b7e8a3407f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:07.338002 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b6cd7-af66-448a-8705-81c4e2a861c9 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:07.338297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b6cd7-af66-448a-8705-81c4e2a861c9 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:07.338581 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b6cd7-af66-448a-8705-81c4e2a861c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:07.360276 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-d99b6cd7-af66-448a-8705-81c4e2a861c9 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:07.361804 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d99b6cd7-af66-448a-8705-81c4e2a861c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:07.362307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b34a0aa3-8b7e-495d-b94a-e855a3c08507 None None] MFA Rules not processed for user `c1d48bafec3045389599887b45ed9ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:07.363788 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b34a0aa3-8b7e-495d-b94a-e855a3c08507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:07.427485 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1052/2138] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:07.440769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b6cd7-af66-448a-8705-81c4e2a861c9 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:07.441260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1087/2139] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:07.441696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d99b6cd7-af66-448a-8705-81c4e2a861c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:07.454553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:07.456499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a0aa3-8b7e-495d-b94a-e855a3c08507 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:07.458069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b34a0aa3-8b7e-495d-b94a-e855a3c08507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:07.523913 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b6cd7-af66-448a-8705-81c4e2a861c9 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:07.529495 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20c1bd-7197-466e-9e15-7039d455e326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:07.530634 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20c1bd-7197-466e-9e15-7039d455e326 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:07.531211 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20c1bd-7197-466e-9e15-7039d455e326 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:07.531650 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20c1bd-7197-466e-9e15-7039d455e326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:07.532933 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a0aa3-8b7e-495d-b94a-e855a3c08507 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:07.537333 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518d87d-66d1-47ee-962f-8c2c2293bb08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:07.538055 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518d87d-66d1-47ee-962f-8c2c2293bb08 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:07.538431 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518d87d-66d1-47ee-962f-8c2c2293bb08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:07.538820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518d87d-66d1-47ee-962f-8c2c2293bb08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:07.543161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20c1bd-7197-466e-9e15-7039d455e326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:07.545284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20c1bd-7197-466e-9e15-7039d455e326 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:07.548334 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518d87d-66d1-47ee-962f-8c2c2293bb08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:07.550038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518d87d-66d1-47ee-962f-8c2c2293bb08 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:07.558775 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1053/2140] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:07.560304 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1088/2141] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:07.583096 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20c1bd-7197-466e-9e15-7039d455e326 service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:07.584737 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-6c20c1bd-7197-466e-9e15-7039d455e326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:07.598761 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20c1bd-7197-466e-9e15-7039d455e326 service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:07.612466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f3578-2a68-48af-ae78-4b42715665ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:07.613136 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f3578-2a68-48af-ae78-4b42715665ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:07.613571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f3578-2a68-48af-ae78-4b42715665ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:07.613922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f3578-2a68-48af-ae78-4b42715665ae admin admin] PATH_INFO: `/v3/users/c1d48bafec3045389599887b45ed9ee4` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:07.615127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f3578-2a68-48af-ae78-4b42715665ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1d48bafec3045389599887b45ed9ee4)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:07.619543 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f3578-2a68-48af-ae78-4b42715665ae admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:07.712896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c80f3578-2a68-48af-ae78-4b42715665ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1d48bafec3045389599887b45ed9ee4'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:07.713897 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c80f3578-2a68-48af-ae78-4b42715665ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1d48bafec3045389599887b45ed9ee4'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:07.714955 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c80f3578-2a68-48af-ae78-4b42715665ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1d48bafec3045389599887b45ed9ee4'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:07.723655 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c80f3578-2a68-48af-ae78-4b42715665ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1d48bafec3045389599887b45ed9ee4'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:07.739025 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c80f3578-2a68-48af-ae78-4b42715665ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1d48bafec3045389599887b45ed9ee4'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:07.784355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1054/2142] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:07 2022] DELETE /identity/v3/users/c1d48bafec3045389599887b45ed9ee4 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:07.789519 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb4ee6-4a70-4a06-a6f8-fb89c824ec46 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:07.790013 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb4ee6-4a70-4a06-a6f8-fb89c824ec46 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:07.790504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb4ee6-4a70-4a06-a6f8-fb89c824ec46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:07.822913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d9bb4ee6-4a70-4a06-a6f8-fb89c824ec46 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:07.824306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d9bb4ee6-4a70-4a06-a6f8-fb89c824ec46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:07.924452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1089/2143] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:07.940690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f3578-2a68-48af-ae78-4b42715665ae admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:07.942356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c80f3578-2a68-48af-ae78-4b42715665ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:08.028091 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb4ee6-4a70-4a06-a6f8-fb89c824ec46 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:08.030099 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d9bb4ee6-4a70-4a06-a6f8-fb89c824ec46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:08.056955 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f3578-2a68-48af-ae78-4b42715665ae admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:08.060928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb4ee6-4a70-4a06-a6f8-fb89c824ec46 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:08.077321 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a74e4-2682-48c9-b325-4a4f9ba7e0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:08.078137 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a74e4-2682-48c9-b325-4a4f9ba7e0be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:08.078316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ee676-d07c-42ea-a7e6-841961b56f72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:08.078641 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a74e4-2682-48c9-b325-4a4f9ba7e0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:08.078956 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ee676-d07c-42ea-a7e6-841961b56f72 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:08.079076 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a74e4-2682-48c9-b325-4a4f9ba7e0be admin admin] PATH_INFO: `/v3/projects/11c86a473d634104bf160997c023fe74` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:08.079240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ee676-d07c-42ea-a7e6-841961b56f72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:08.079517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ee676-d07c-42ea-a7e6-841961b56f72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:08.080680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a74e4-2682-48c9-b325-4a4f9ba7e0be admin admin] RBAC: Authorizing `identity:delete_project(project_id=11c86a473d634104bf160997c023fe74)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:08.083545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a74e4-2682-48c9-b325-4a4f9ba7e0be admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:08.086035 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:08.097544 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ee676-d07c-42ea-a7e6-841961b56f72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:08.099169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ee676-d07c-42ea-a7e6-841961b56f72 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:08.127270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1055/2144] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:08.180063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-240a74e4-2682-48c9-b325-4a4f9ba7e0be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11c86a473d634104bf160997c023fe74', 'request_id': 'req-240a74e4-2682-48c9-b325-4a4f9ba7e0be'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.186250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-240a74e4-2682-48c9-b325-4a4f9ba7e0be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11c86a473d634104bf160997c023fe74', 'request_id': 'req-240a74e4-2682-48c9-b325-4a4f9ba7e0be'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.196819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-240a74e4-2682-48c9-b325-4a4f9ba7e0be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11c86a473d634104bf160997c023fe74', 'request_id': 'req-240a74e4-2682-48c9-b325-4a4f9ba7e0be'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.225674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-240a74e4-2682-48c9-b325-4a4f9ba7e0be admin admin] The token cache is being invalidate because project 11c86a473d634104bf160997c023fe74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:08.225674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-240a74e4-2682-48c9-b325-4a4f9ba7e0be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.225674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-240a74e4-2682-48c9-b325-4a4f9ba7e0be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.225674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1090/2145] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:08 2022] DELETE /identity/v3/projects/11c86a473d634104bf160997c023fe74 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:08.228766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ee676-d07c-42ea-a7e6-841961b56f72 service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:08.244083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ee676-d07c-42ea-a7e6-841961b56f72 service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:08.281657 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:08.334220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51ac8c-559c-4cee-8a5f-ad52f250f2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:08.335800 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51ac8c-559c-4cee-8a5f-ad52f250f2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:08.336525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51ac8c-559c-4cee-8a5f-ad52f250f2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:08.337174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51ac8c-559c-4cee-8a5f-ad52f250f2bd admin admin] PATH_INFO: `/v3/users/3dfa7f500871407c8a66bb550ee49140` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:08.339332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51ac8c-559c-4cee-8a5f-ad52f250f2bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dfa7f500871407c8a66bb550ee49140)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:08.341663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51ac8c-559c-4cee-8a5f-ad52f250f2bd admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:08.407084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-4b51ac8c-559c-4cee-8a5f-ad52f250f2bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dfa7f500871407c8a66bb550ee49140'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.408028 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-4b51ac8c-559c-4cee-8a5f-ad52f250f2bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dfa7f500871407c8a66bb550ee49140'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.408802 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-4b51ac8c-559c-4cee-8a5f-ad52f250f2bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dfa7f500871407c8a66bb550ee49140'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.417164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-4b51ac8c-559c-4cee-8a5f-ad52f250f2bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dfa7f500871407c8a66bb550ee49140'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.426874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-4b51ac8c-559c-4cee-8a5f-ad52f250f2bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dfa7f500871407c8a66bb550ee49140'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.452729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1056/2146] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:08 2022] DELETE /identity/v3/users/3dfa7f500871407c8a66bb550ee49140 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:08.467558 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a74e4-2682-48c9-b325-4a4f9ba7e0be admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:08.469926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-240a74e4-2682-48c9-b325-4a4f9ba7e0be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:08.495616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a74e4-2682-48c9-b325-4a4f9ba7e0be admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:08.587903 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:08.589002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:08.589002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:08.589197 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae admin admin] PATH_INFO: `/v3/projects/e5949afd7d8b483db23f203952fc0547` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:08.590184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5949afd7d8b483db23f203952fc0547)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:08.592133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:08.660121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5949afd7d8b483db23f203952fc0547', 'request_id': 'req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.666509 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5949afd7d8b483db23f203952fc0547', 'request_id': 'req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.678393 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5949afd7d8b483db23f203952fc0547', 'request_id': 'req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.692223 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae admin admin] The token cache is being invalidate because project e5949afd7d8b483db23f203952fc0547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:08.692677 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.693348 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.695021 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1091/2147] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:08 2022] DELETE /identity/v3/projects/e5949afd7d8b483db23f203952fc0547 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:08.700080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecf905-c492-4fe8-931e-ba34f0da2c98 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:08.700448 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecf905-c492-4fe8-931e-ba34f0da2c98 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:08.700816 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecf905-c492-4fe8-931e-ba34f0da2c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:08.730314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-d2ecf905-c492-4fe8-931e-ba34f0da2c98 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:08.731526 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d2ecf905-c492-4fe8-931e-ba34f0da2c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:08.777518 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:08.779022 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:08.825040 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1057/2148] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:08.827904 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:08.833011 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352adc9c-9112-473d-abe3-97193a59e33d None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:08.833303 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352adc9c-9112-473d-abe3-97193a59e33d None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:08.833707 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352adc9c-9112-473d-abe3-97193a59e33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:08.854840 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-352adc9c-9112-473d-abe3-97193a59e33d None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:08.856102 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-352adc9c-9112-473d-abe3-97193a59e33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:08.869423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1058/2149] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:08.870203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352adc9c-9112-473d-abe3-97193a59e33d None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:08.877477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ba6d6-fc0f-433b-8c3e-6de1ca7984ae admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:08.880063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352adc9c-9112-473d-abe3-97193a59e33d None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:08.884752 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee1b62-1646-48a2-be9a-6cf523959303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:08.884899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053442a-a0a5-4cd9-83eb-ea16ec506895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:08.885465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053442a-a0a5-4cd9-83eb-ea16ec506895 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:08.885582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee1b62-1646-48a2-be9a-6cf523959303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:08.885745 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053442a-a0a5-4cd9-83eb-ea16ec506895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:08.885859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee1b62-1646-48a2-be9a-6cf523959303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:08.886018 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053442a-a0a5-4cd9-83eb-ea16ec506895 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:08.886122 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee1b62-1646-48a2-be9a-6cf523959303 admin admin] PATH_INFO: `/v3/projects/ceb4c7fc27684188bf814286191c9aa9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:08.886858 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053442a-a0a5-4cd9-83eb-ea16ec506895 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:08.887164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053442a-a0a5-4cd9-83eb-ea16ec506895 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:08.887414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee1b62-1646-48a2-be9a-6cf523959303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceb4c7fc27684188bf814286191c9aa9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:08.888793 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053442a-a0a5-4cd9-83eb-ea16ec506895 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:08.889370 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee1b62-1646-48a2-be9a-6cf523959303 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:08.905955 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1059/2150] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:34:08.910295 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053442a-a0a5-4cd9-83eb-ea16ec506895 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:08.922131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053442a-a0a5-4cd9-83eb-ea16ec506895 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:08.928176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392641fe-80b7-4d6f-b0dd-ef4b44328891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:08.928980 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392641fe-80b7-4d6f-b0dd-ef4b44328891 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:08.929424 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392641fe-80b7-4d6f-b0dd-ef4b44328891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:08.929860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392641fe-80b7-4d6f-b0dd-ef4b44328891 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:08.930682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392641fe-80b7-4d6f-b0dd-ef4b44328891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:08.932846 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392641fe-80b7-4d6f-b0dd-ef4b44328891 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:08.944485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-afee1b62-1646-48a2-be9a-6cf523959303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceb4c7fc27684188bf814286191c9aa9', 'request_id': 'req-afee1b62-1646-48a2-be9a-6cf523959303'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.951707 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-afee1b62-1646-48a2-be9a-6cf523959303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceb4c7fc27684188bf814286191c9aa9', 'request_id': 'req-afee1b62-1646-48a2-be9a-6cf523959303'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.960377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-afee1b62-1646-48a2-be9a-6cf523959303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceb4c7fc27684188bf814286191c9aa9', 'request_id': 'req-afee1b62-1646-48a2-be9a-6cf523959303'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.965066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1060/2151] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:08 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:34:08.970127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392641fe-80b7-4d6f-b0dd-ef4b44328891 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:08.976084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:08.982845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-afee1b62-1646-48a2-be9a-6cf523959303 admin admin] The token cache is being invalidate because project ceb4c7fc27684188bf814286191c9aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:08.983153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-afee1b62-1646-48a2-be9a-6cf523959303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.984052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-afee1b62-1646-48a2-be9a-6cf523959303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:08.989760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1092/2152] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:08 2022] DELETE /identity/v3/projects/ceb4c7fc27684188bf814286191c9aa9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:08.991526 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80882f-c15d-4f6b-a5fd-5ee7da4db6af None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:08.991899 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80882f-c15d-4f6b-a5fd-5ee7da4db6af None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:08.992182 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80882f-c15d-4f6b-a5fd-5ee7da4db6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.012419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8b80882f-c15d-4f6b-a5fd-5ee7da4db6af None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:09.013636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8b80882f-c15d-4f6b-a5fd-5ee7da4db6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:09.051432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392641fe-80b7-4d6f-b0dd-ef4b44328891 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:09.060686 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3135baa-f669-44e6-aff2-f7ed4e56a0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:09.061179 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3135baa-f669-44e6-aff2-f7ed4e56a0db admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:09.062150 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3135baa-f669-44e6-aff2-f7ed4e56a0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:09.062546 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3135baa-f669-44e6-aff2-f7ed4e56a0db admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.063286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3135baa-f669-44e6-aff2-f7ed4e56a0db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:09.065277 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3135baa-f669-44e6-aff2-f7ed4e56a0db admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:09.084243 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1093/2153] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:09.089902 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1061/2154] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:08 2022] POST /identity/v3/users => generated 282 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:34:09.092076 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b6dec-903e-4e62-9a30-aca92f32dbb8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:09.092810 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b6dec-903e-4e62-9a30-aca92f32dbb8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:09.093175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b6dec-903e-4e62-9a30-aca92f32dbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.093765 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3135baa-f669-44e6-aff2-f7ed4e56a0db admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:09.095449 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e3135baa-f669-44e6-aff2-f7ed4e56a0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:09.108227 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3135baa-f669-44e6-aff2-f7ed4e56a0db admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:09.113634 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16adb00-99a4-4ac0-8e33-02e2428ff32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:09.114299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16adb00-99a4-4ac0-8e33-02e2428ff32b admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:09.114587 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16adb00-99a4-4ac0-8e33-02e2428ff32b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:09.114862 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16adb00-99a4-4ac0-8e33-02e2428ff32b admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.115549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16adb00-99a4-4ac0-8e33-02e2428ff32b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:09.116782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-108b6dec-903e-4e62-9a30-aca92f32dbb8 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:09.117245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16adb00-99a4-4ac0-8e33-02e2428ff32b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:09.117989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-108b6dec-903e-4e62-9a30-aca92f32dbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:09.144081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1094/2155] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:09.146758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1062/2156] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:09.147967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b6dec-903e-4e62-9a30-aca92f32dbb8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:09.148702 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16adb00-99a4-4ac0-8e33-02e2428ff32b admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:09.161312 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16adb00-99a4-4ac0-8e33-02e2428ff32b admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:09.161935 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b6dec-903e-4e62-9a30-aca92f32dbb8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:09.167517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eefa140-0aaa-40e3-b9d7-40b962ae2aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:09.167757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4e220-89e8-4a27-b370-a6a991f598ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:09.168351 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefa140-0aaa-40e3-b9d7-40b962ae2aad admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:09.168483 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4e220-89e8-4a27-b370-a6a991f598ba admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:09.168636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4e220-89e8-4a27-b370-a6a991f598ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:09.168907 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefa140-0aaa-40e3-b9d7-40b962ae2aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:09.169023 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4e220-89e8-4a27-b370-a6a991f598ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.169374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefa140-0aaa-40e3-b9d7-40b962ae2aad admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.171925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eefa140-0aaa-40e3-b9d7-40b962ae2aad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:09.172770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4e220-89e8-4a27-b370-a6a991f598ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:09.172860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eefa140-0aaa-40e3-b9d7-40b962ae2aad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:09.174108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eefa140-0aaa-40e3-b9d7-40b962ae2aad admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:09.190958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4e220-89e8-4a27-b370-a6a991f598ba admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:09.197653 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1063/2157] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:34:09.200625 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eefa140-0aaa-40e3-b9d7-40b962ae2aad admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:09.212248 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eefa140-0aaa-40e3-b9d7-40b962ae2aad admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:09.218056 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e3c78-7d2f-4b93-8a07-401ff1f2f690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:09.218999 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e3c78-7d2f-4b93-8a07-401ff1f2f690 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:09.219461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e3c78-7d2f-4b93-8a07-401ff1f2f690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:09.219882 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239e3c78-7d2f-4b93-8a07-401ff1f2f690 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.220616 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e3c78-7d2f-4b93-8a07-401ff1f2f690 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:09.222645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239e3c78-7d2f-4b93-8a07-401ff1f2f690 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:09.227962 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1095/2158] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:09.228959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4e220-89e8-4a27-b370-a6a991f598ba admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:09.244839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4e220-89e8-4a27-b370-a6a991f598ba admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:09.253501 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035010c0-c9f4-4836-89cc-5123d7898a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:09.253501 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035010c0-c9f4-4836-89cc-5123d7898a95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:09.253903 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035010c0-c9f4-4836-89cc-5123d7898a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:09.254413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035010c0-c9f4-4836-89cc-5123d7898a95 admin admin] PATH_INFO: `/v3/projects/910593fbebc941919de6d26feae6cac6/users/95680494f6e54d8f83467f52be785c1b/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.287200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1064/2159] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:09 2022] POST /identity/v3/projects => generated 381 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:09.287440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035010c0-c9f4-4836-89cc-5123d7898a95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=910593fbebc941919de6d26feae6cac6, user_id=95680494f6e54d8f83467f52be785c1b, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:09.288224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035010c0-c9f4-4836-89cc-5123d7898a95 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:09.288707 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e3c78-7d2f-4b93-8a07-401ff1f2f690 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:09.315662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1096/2160] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:09 2022] PUT /identity/v3/projects/910593fbebc941919de6d26feae6cac6/users/95680494f6e54d8f83467f52be785c1b/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:09.320020 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db082-e6b9-40fa-b4ac-6e8c2f56f3a6 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:09.320310 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db082-e6b9-40fa-b4ac-6e8c2f56f3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:09.320592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db082-e6b9-40fa-b4ac-6e8c2f56f3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.341855 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:09.375589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-989db082-e6b9-40fa-b4ac-6e8c2f56f3a6 None None] MFA Rules not processed for user `95680494f6e54d8f83467f52be785c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:09.377570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-989db082-e6b9-40fa-b4ac-6e8c2f56f3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:09.392169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239e3c78-7d2f-4b93-8a07-401ff1f2f690 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:09.472391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:09.482121 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e644405-0fc7-4355-adff-1ec2c739c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:09.482947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e644405-0fc7-4355-adff-1ec2c739c442 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:09.483447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e644405-0fc7-4355-adff-1ec2c739c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:09.483885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e644405-0fc7-4355-adff-1ec2c739c442 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.484460 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e644405-0fc7-4355-adff-1ec2c739c442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:09.486613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1097/2161] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:09.487108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e644405-0fc7-4355-adff-1ec2c739c442 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:09.499544 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989db082-e6b9-40fa-b4ac-6e8c2f56f3a6 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:09.503107 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-989db082-e6b9-40fa-b4ac-6e8c2f56f3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:09.525853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1065/2162] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:09 2022] POST /identity/v3/users => generated 291 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:09.530086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e644405-0fc7-4355-adff-1ec2c739c442 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:09.543675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e644405-0fc7-4355-adff-1ec2c739c442 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:09.556927 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747eacc-481f-4e5a-bbcf-3cea93700dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:09.563259 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747eacc-481f-4e5a-bbcf-3cea93700dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:09.563553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747eacc-481f-4e5a-bbcf-3cea93700dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:09.564449 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747eacc-481f-4e5a-bbcf-3cea93700dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.568029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747eacc-481f-4e5a-bbcf-3cea93700dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:09.571215 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747eacc-481f-4e5a-bbcf-3cea93700dd4 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:09.589502 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1066/2163] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:34:09.593401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747eacc-481f-4e5a-bbcf-3cea93700dd4 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:09.593544 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989db082-e6b9-40fa-b4ac-6e8c2f56f3a6 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:09.597448 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a5964-36ba-464e-b7cf-331546b2fc29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:09.598143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a5964-36ba-464e-b7cf-331546b2fc29 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:09.598511 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a5964-36ba-464e-b7cf-331546b2fc29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:09.599084 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a5964-36ba-464e-b7cf-331546b2fc29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.604208 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747eacc-481f-4e5a-bbcf-3cea93700dd4 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:09.609034 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ed484-b764-4364-bab4-1bf9fbf778cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:09.609640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ed484-b764-4364-bab4-1bf9fbf778cf admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:09.609924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ed484-b764-4364-bab4-1bf9fbf778cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:09.610194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ed484-b764-4364-bab4-1bf9fbf778cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.610838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ed484-b764-4364-bab4-1bf9fbf778cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:09.611109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a5964-36ba-464e-b7cf-331546b2fc29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:09.612475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ed484-b764-4364-bab4-1bf9fbf778cf admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:09.613244 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a5964-36ba-464e-b7cf-331546b2fc29 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:09.629660 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1067/2164] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:09.630572 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ed484-b764-4364-bab4-1bf9fbf778cf admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:09.636864 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1098/2165] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:09.642494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ed484-b764-4364-bab4-1bf9fbf778cf admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:09.670994 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2344f-87b1-490e-a1f8-7ed6f28a0b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:09.672143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2344f-87b1-490e-a1f8-7ed6f28a0b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:09.672658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2344f-87b1-490e-a1f8-7ed6f28a0b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:09.673183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2344f-87b1-490e-a1f8-7ed6f28a0b6d admin admin] PATH_INFO: `/v3/projects/412e1c8bf44048b4bc2940ce0fe22b4a/users/f179ee45c8e247e6815f1981802a0c32/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.708272 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2344f-87b1-490e-a1f8-7ed6f28a0b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=412e1c8bf44048b4bc2940ce0fe22b4a, user_id=f179ee45c8e247e6815f1981802a0c32, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:09.711671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2344f-87b1-490e-a1f8-7ed6f28a0b6d admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:09.731454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1068/2166] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:09 2022] PUT /identity/v3/projects/412e1c8bf44048b4bc2940ce0fe22b4a/users/f179ee45c8e247e6815f1981802a0c32/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:09.742353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3568fb-2e21-4af2-b90e-babb89b37130 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:09.742644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3568fb-2e21-4af2-b90e-babb89b37130 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:09.742926 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3568fb-2e21-4af2-b90e-babb89b37130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.778143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fb3568fb-2e21-4af2-b90e-babb89b37130 None None] MFA Rules not processed for user `f179ee45c8e247e6815f1981802a0c32`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:09.779346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fb3568fb-2e21-4af2-b90e-babb89b37130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:09.866410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1099/2167] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:09.868592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2344f-87b1-490e-a1f8-7ed6f28a0b6d admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:09.952540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2344f-87b1-490e-a1f8-7ed6f28a0b6d admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:09.958057 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804ac6e-b8fa-4844-a2be-00cdc70194bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:09.958755 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804ac6e-b8fa-4844-a2be-00cdc70194bc admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:09.959139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804ac6e-b8fa-4844-a2be-00cdc70194bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:09.959741 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804ac6e-b8fa-4844-a2be-00cdc70194bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:09.960436 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804ac6e-b8fa-4844-a2be-00cdc70194bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:09.962558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804ac6e-b8fa-4844-a2be-00cdc70194bc admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:09.999889 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1069/2168] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:09 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:10.001024 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3568fb-2e21-4af2-b90e-babb89b37130 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:10.012763 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3568fb-2e21-4af2-b90e-babb89b37130 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:10.051898 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:10.080948 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a15bf-2cc4-4bb0-a966-92dccae2fc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:10.081553 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a15bf-2cc4-4bb0-a966-92dccae2fc0e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.081831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a15bf-2cc4-4bb0-a966-92dccae2fc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.082100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a15bf-2cc4-4bb0-a966-92dccae2fc0e admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.082734 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a15bf-2cc4-4bb0-a966-92dccae2fc0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:10.084731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a15bf-2cc4-4bb0-a966-92dccae2fc0e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:10.105955 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1100/2169] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:09 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:10.109816 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804ac6e-b8fa-4844-a2be-00cdc70194bc admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:10.119851 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804ac6e-b8fa-4844-a2be-00cdc70194bc admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:10.124446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd53f8-14a9-46ce-b690-72d888834a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:10.125032 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd53f8-14a9-46ce-b690-72d888834a05 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.125310 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd53f8-14a9-46ce-b690-72d888834a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.125581 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd53f8-14a9-46ce-b690-72d888834a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.126221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd53f8-14a9-46ce-b690-72d888834a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:10.127838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd53f8-14a9-46ce-b690-72d888834a05 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:10.138083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1070/2170] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:10.143853 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a15bf-2cc4-4bb0-a966-92dccae2fc0e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:10.154653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a15bf-2cc4-4bb0-a966-92dccae2fc0e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:10.160231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06a91b-66cb-4c90-a3c2-696bd7e8be08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:10.160888 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06a91b-66cb-4c90-a3c2-696bd7e8be08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.161171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06a91b-66cb-4c90-a3c2-696bd7e8be08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.161454 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06a91b-66cb-4c90-a3c2-696bd7e8be08 admin admin] PATH_INFO: `/v3/projects/0ad35a2fa7f24f46aaf010c78a81ebd2/users/d2e6e0d118f94dc2b88d9728dbf1d38f/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.176307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06a91b-66cb-4c90-a3c2-696bd7e8be08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad35a2fa7f24f46aaf010c78a81ebd2, user_id=d2e6e0d118f94dc2b88d9728dbf1d38f, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:10.179018 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be06a91b-66cb-4c90-a3c2-696bd7e8be08 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:10.192303 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1101/2171] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:10 2022] PUT /identity/v3/projects/0ad35a2fa7f24f46aaf010c78a81ebd2/users/d2e6e0d118f94dc2b88d9728dbf1d38f/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:10.196310 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd53f8-14a9-46ce-b690-72d888834a05 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:10.207116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd53f8-14a9-46ce-b690-72d888834a05 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:10.266630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6afff-3ed8-41e1-81b4-aad0060956e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:10.267247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6afff-3ed8-41e1-81b4-aad0060956e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.267584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6afff-3ed8-41e1-81b4-aad0060956e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.267936 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6afff-3ed8-41e1-81b4-aad0060956e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.268628 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6afff-3ed8-41e1-81b4-aad0060956e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:10.270644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6afff-3ed8-41e1-81b4-aad0060956e6 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:10.281363 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1071/2172] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:10.285344 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06a91b-66cb-4c90-a3c2-696bd7e8be08 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:10.296586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06a91b-66cb-4c90-a3c2-696bd7e8be08 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:10.302012 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e189a-18c6-460b-8b37-3193b34d05f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:10.302732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e189a-18c6-460b-8b37-3193b34d05f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.303015 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e189a-18c6-460b-8b37-3193b34d05f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.303290 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e189a-18c6-460b-8b37-3193b34d05f2 admin admin] PATH_INFO: `/v3/projects/0ad35a2fa7f24f46aaf010c78a81ebd2/users/d2e6e0d118f94dc2b88d9728dbf1d38f/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.305480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e189a-18c6-460b-8b37-3193b34d05f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ad35a2fa7f24f46aaf010c78a81ebd2, user_id=d2e6e0d118f94dc2b88d9728dbf1d38f, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:10.308286 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e189a-18c6-460b-8b37-3193b34d05f2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:10.320147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1102/2173] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:10 2022] PUT /identity/v3/projects/0ad35a2fa7f24f46aaf010c78a81ebd2/users/d2e6e0d118f94dc2b88d9728dbf1d38f/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:10.333109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b54a3c-35d2-4b10-a704-dbd90dd6b23f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.333399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b54a3c-35d2-4b10-a704-dbd90dd6b23f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.333672 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b54a3c-35d2-4b10-a704-dbd90dd6b23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.353871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:10.364296 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-82b54a3c-35d2-4b10-a704-dbd90dd6b23f None None] MFA Rules not processed for user `d2e6e0d118f94dc2b88d9728dbf1d38f`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:10.365505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-82b54a3c-35d2-4b10-a704-dbd90dd6b23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:10.434305 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1072/2174] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:10.447266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e189a-18c6-460b-8b37-3193b34d05f2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:10.509174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e189a-18c6-460b-8b37-3193b34d05f2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:10.513078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878b03c-3fed-4f32-b97f-e76a4d28b08a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:10.513770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878b03c-3fed-4f32-b97f-e76a4d28b08a service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.514140 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878b03c-3fed-4f32-b97f-e76a4d28b08a service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.515165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878b03c-3fed-4f32-b97f-e76a4d28b08a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.524711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b878b03c-3fed-4f32-b97f-e76a4d28b08a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:10.526219 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b878b03c-3fed-4f32-b97f-e76a4d28b08a service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:10.536365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1103/2175] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:10 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:10.910374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b54a3c-35d2-4b10-a704-dbd90dd6b23f None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:10.919657 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b54a3c-35d2-4b10-a704-dbd90dd6b23f None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:10.926066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43617c2a-0ed9-4520-9521-bba4b1c6d71d tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2e6e0d118f94dc2b88d9728dbf1d38f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad35a2fa7f24f46aaf010c78a81ebd2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:10.926759 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43617c2a-0ed9-4520-9521-bba4b1c6d71d tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.927118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43617c2a-0ed9-4520-9521-bba4b1c6d71d tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.927470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43617c2a-0ed9-4520-9521-bba4b1c6d71d tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.928278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43617c2a-0ed9-4520-9521-bba4b1c6d71d tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:10.928664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43617c2a-0ed9-4520-9521-bba4b1c6d71d tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:10.930379 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43617c2a-0ed9-4520-9521-bba4b1c6d71d tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:10.944229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1073/2176] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:34:10.948041 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878b03c-3fed-4f32-b97f-e76a4d28b08a service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:10.958846 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878b03c-3fed-4f32-b97f-e76a4d28b08a service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:10.963954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1814b-d611-4237-8040-9e6cf6d655d2 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2e6e0d118f94dc2b88d9728dbf1d38f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad35a2fa7f24f46aaf010c78a81ebd2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:10.964341 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1814b-d611-4237-8040-9e6cf6d655d2 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:10.964720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1814b-d611-4237-8040-9e6cf6d655d2 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:10.965113 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1814b-d611-4237-8040-9e6cf6d655d2 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:10.965848 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1814b-d611-4237-8040-9e6cf6d655d2 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:10.968218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1814b-d611-4237-8040-9e6cf6d655d2 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:10.998628 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1104/2177] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:10 2022] POST /identity/v3/projects => generated 354 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:11.004369 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43617c2a-0ed9-4520-9521-bba4b1c6d71d tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:11.015607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43617c2a-0ed9-4520-9521-bba4b1c6d71d tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:11.086150 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a40e5-7346-4178-a27f-5d8b20dbd0a6 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2e6e0d118f94dc2b88d9728dbf1d38f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad35a2fa7f24f46aaf010c78a81ebd2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:11.086788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a40e5-7346-4178-a27f-5d8b20dbd0a6 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:11.087069 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a40e5-7346-4178-a27f-5d8b20dbd0a6 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:11.087340 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a40e5-7346-4178-a27f-5d8b20dbd0a6 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:11.088025 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a40e5-7346-4178-a27f-5d8b20dbd0a6 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:11.088333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a40e5-7346-4178-a27f-5d8b20dbd0a6 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:11.090035 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a40e5-7346-4178-a27f-5d8b20dbd0a6 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:11.105419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1074/2178] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:34:11.154254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1814b-d611-4237-8040-9e6cf6d655d2 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:11.165927 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1814b-d611-4237-8040-9e6cf6d655d2 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:11.171857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7209eb-1700-4622-97f9-acfb272a0a39 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2e6e0d118f94dc2b88d9728dbf1d38f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0ad35a2fa7f24f46aaf010c78a81ebd2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:11.172810 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7209eb-1700-4622-97f9-acfb272a0a39 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:11.173111 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7209eb-1700-4622-97f9-acfb272a0a39 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:11.173550 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7209eb-1700-4622-97f9-acfb272a0a39 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] PATH_INFO: `/v3/projects/109fb71b51c14380bf77b1d0474565b4` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:11.176353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7209eb-1700-4622-97f9-acfb272a0a39 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] RBAC: Authorizing `identity:delete_project(project_id=109fb71b51c14380bf77b1d0474565b4)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:11.178401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7209eb-1700-4622-97f9-acfb272a0a39 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:11.231633 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6f7209eb-1700-4622-97f9-acfb272a0a39 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '109fb71b51c14380bf77b1d0474565b4', 'request_id': 'req-6f7209eb-1700-4622-97f9-acfb272a0a39'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:11.239075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6f7209eb-1700-4622-97f9-acfb272a0a39 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '109fb71b51c14380bf77b1d0474565b4', 'request_id': 'req-6f7209eb-1700-4622-97f9-acfb272a0a39'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:11.240965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:DELETE=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:11.248537 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6f7209eb-1700-4622-97f9-acfb272a0a39 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '109fb71b51c14380bf77b1d0474565b4', 'request_id': 'req-6f7209eb-1700-4622-97f9-acfb272a0a39'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:11.264493 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6f7209eb-1700-4622-97f9-acfb272a0a39 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] The token cache is being invalidate because project 109fb71b51c14380bf77b1d0474565b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:11.264783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6f7209eb-1700-4622-97f9-acfb272a0a39 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:11.265492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6f7209eb-1700-4622-97f9-acfb272a0a39 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:11.267195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1105/2179] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:11 2022] DELETE /identity/v3/projects/109fb71b51c14380bf77b1d0474565b4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:11.479123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a40e5-7346-4178-a27f-5d8b20dbd0a6 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:11.480927 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-215a40e5-7346-4178-a27f-5d8b20dbd0a6 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:11.553382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a40e5-7346-4178-a27f-5d8b20dbd0a6 tempest-VolumeQuotaClassesTest-803185340 tempest-VolumeQuotaClassesTest-803185340-project-admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:11.571698 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:11.614613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84c4a4-03d4-4814-8ef2-95f965d1dc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:11.615333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84c4a4-03d4-4814-8ef2-95f965d1dc2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:11.615752 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84c4a4-03d4-4814-8ef2-95f965d1dc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:11.616007 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84c4a4-03d4-4814-8ef2-95f965d1dc2e admin admin] PATH_INFO: `/v3/users/f179ee45c8e247e6815f1981802a0c32` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:11.617174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84c4a4-03d4-4814-8ef2-95f965d1dc2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f179ee45c8e247e6815f1981802a0c32)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:11.619144 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84c4a4-03d4-4814-8ef2-95f965d1dc2e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:11.667204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-fc84c4a4-03d4-4814-8ef2-95f965d1dc2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f179ee45c8e247e6815f1981802a0c32'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:11.667940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-fc84c4a4-03d4-4814-8ef2-95f965d1dc2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f179ee45c8e247e6815f1981802a0c32'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:11.668593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-fc84c4a4-03d4-4814-8ef2-95f965d1dc2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f179ee45c8e247e6815f1981802a0c32'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:11.675492 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-fc84c4a4-03d4-4814-8ef2-95f965d1dc2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f179ee45c8e247e6815f1981802a0c32'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:11.690884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-fc84c4a4-03d4-4814-8ef2-95f965d1dc2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f179ee45c8e247e6815f1981802a0c32'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:11.715181 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1075/2180] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:11 2022] DELETE /identity/v3/users/f179ee45c8e247e6815f1981802a0c32 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:11.720106 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f894c1-75cd-4129-b51b-eaac5eb8ad53 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:11.720497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f894c1-75cd-4129-b51b-eaac5eb8ad53 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:11.720644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f894c1-75cd-4129-b51b-eaac5eb8ad53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:11.740120 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f2f894c1-75cd-4129-b51b-eaac5eb8ad53 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:11.741360 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f2f894c1-75cd-4129-b51b-eaac5eb8ad53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:11.815147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1106/2181] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:11.824999 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84c4a4-03d4-4814-8ef2-95f965d1dc2e admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:11.826430 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fc84c4a4-03d4-4814-8ef2-95f965d1dc2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:11.906224 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84c4a4-03d4-4814-8ef2-95f965d1dc2e admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:11.910710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c9cb2-6f1e-41ea-9ea9-2418af52a97e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:11.911410 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c9cb2-6f1e-41ea-9ea9-2418af52a97e service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:11.911723 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c9cb2-6f1e-41ea-9ea9-2418af52a97e service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:11.912006 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c9cb2-6f1e-41ea-9ea9-2418af52a97e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:11.921869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c9cb2-6f1e-41ea-9ea9-2418af52a97e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:11.923310 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c9cb2-6f1e-41ea-9ea9-2418af52a97e service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:11.933696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1076/2182] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:12.222800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f894c1-75cd-4129-b51b-eaac5eb8ad53 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:12.224368 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f2f894c1-75cd-4129-b51b-eaac5eb8ad53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:12.238152 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f894c1-75cd-4129-b51b-eaac5eb8ad53 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:12.242916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a896cf8-88cf-4c8f-be2b-cbae04c891db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:12.243565 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a896cf8-88cf-4c8f-be2b-cbae04c891db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:12.243884 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a896cf8-88cf-4c8f-be2b-cbae04c891db admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:12.244164 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a896cf8-88cf-4c8f-be2b-cbae04c891db admin admin] PATH_INFO: `/v3/projects/412e1c8bf44048b4bc2940ce0fe22b4a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:12.245416 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a896cf8-88cf-4c8f-be2b-cbae04c891db admin admin] RBAC: Authorizing `identity:delete_project(project_id=412e1c8bf44048b4bc2940ce0fe22b4a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:12.247368 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a896cf8-88cf-4c8f-be2b-cbae04c891db admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:12.315389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a896cf8-88cf-4c8f-be2b-cbae04c891db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '412e1c8bf44048b4bc2940ce0fe22b4a', 'request_id': 'req-6a896cf8-88cf-4c8f-be2b-cbae04c891db'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:12.322127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a896cf8-88cf-4c8f-be2b-cbae04c891db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '412e1c8bf44048b4bc2940ce0fe22b4a', 'request_id': 'req-6a896cf8-88cf-4c8f-be2b-cbae04c891db'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:12.330185 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a896cf8-88cf-4c8f-be2b-cbae04c891db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '412e1c8bf44048b4bc2940ce0fe22b4a', 'request_id': 'req-6a896cf8-88cf-4c8f-be2b-cbae04c891db'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:12.341859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a896cf8-88cf-4c8f-be2b-cbae04c891db admin admin] The token cache is being invalidate because project 412e1c8bf44048b4bc2940ce0fe22b4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:12.342147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a896cf8-88cf-4c8f-be2b-cbae04c891db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:12.342805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a896cf8-88cf-4c8f-be2b-cbae04c891db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:12.344395 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1107/2183] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:12 2022] DELETE /identity/v3/projects/412e1c8bf44048b4bc2940ce0fe22b4a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:12.348445 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c9cb2-6f1e-41ea-9ea9-2418af52a97e service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:12.363186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c9cb2-6f1e-41ea-9ea9-2418af52a97e service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:12.410507 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:12.428229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396de14-ab79-4c86-8d8a-a33bd412ada2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:12.428862 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396de14-ab79-4c86-8d8a-a33bd412ada2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:12.429163 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396de14-ab79-4c86-8d8a-a33bd412ada2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:12.429426 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396de14-ab79-4c86-8d8a-a33bd412ada2 admin admin] PATH_INFO: `/v3/users/d2e6e0d118f94dc2b88d9728dbf1d38f` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:12.430545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396de14-ab79-4c86-8d8a-a33bd412ada2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2e6e0d118f94dc2b88d9728dbf1d38f)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:12.432516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396de14-ab79-4c86-8d8a-a33bd412ada2 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:12.477601 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1396de14-ab79-4c86-8d8a-a33bd412ada2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2e6e0d118f94dc2b88d9728dbf1d38f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:12.478234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1396de14-ab79-4c86-8d8a-a33bd412ada2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2e6e0d118f94dc2b88d9728dbf1d38f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:12.478821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1396de14-ab79-4c86-8d8a-a33bd412ada2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2e6e0d118f94dc2b88d9728dbf1d38f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:12.486466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1396de14-ab79-4c86-8d8a-a33bd412ada2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2e6e0d118f94dc2b88d9728dbf1d38f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:12.493699 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1396de14-ab79-4c86-8d8a-a33bd412ada2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2e6e0d118f94dc2b88d9728dbf1d38f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:12.511153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1077/2184] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:12 2022] DELETE /identity/v3/users/d2e6e0d118f94dc2b88d9728dbf1d38f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:12.796954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a896cf8-88cf-4c8f-be2b-cbae04c891db admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:12.798439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6a896cf8-88cf-4c8f-be2b-cbae04c891db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:12.821363 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a896cf8-88cf-4c8f-be2b-cbae04c891db admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:12.895644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c758e-ebd1-4a2a-91ad-1929beae2f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:12.896315 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c758e-ebd1-4a2a-91ad-1929beae2f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:12.896599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c758e-ebd1-4a2a-91ad-1929beae2f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:12.896879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c758e-ebd1-4a2a-91ad-1929beae2f1c admin admin] PATH_INFO: `/v3/projects/0ad35a2fa7f24f46aaf010c78a81ebd2` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:12.898051 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c758e-ebd1-4a2a-91ad-1929beae2f1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ad35a2fa7f24f46aaf010c78a81ebd2)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:12.900021 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c758e-ebd1-4a2a-91ad-1929beae2f1c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:12.914611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:12.960471 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7b9c758e-ebd1-4a2a-91ad-1929beae2f1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ad35a2fa7f24f46aaf010c78a81ebd2', 'request_id': 'req-7b9c758e-ebd1-4a2a-91ad-1929beae2f1c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:12.966976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7b9c758e-ebd1-4a2a-91ad-1929beae2f1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ad35a2fa7f24f46aaf010c78a81ebd2', 'request_id': 'req-7b9c758e-ebd1-4a2a-91ad-1929beae2f1c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:12.973776 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7b9c758e-ebd1-4a2a-91ad-1929beae2f1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ad35a2fa7f24f46aaf010c78a81ebd2', 'request_id': 'req-7b9c758e-ebd1-4a2a-91ad-1929beae2f1c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:12.990788 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7b9c758e-ebd1-4a2a-91ad-1929beae2f1c admin admin] The token cache is being invalidate because project 0ad35a2fa7f24f46aaf010c78a81ebd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:12.991124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7b9c758e-ebd1-4a2a-91ad-1929beae2f1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:12.992010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7b9c758e-ebd1-4a2a-91ad-1929beae2f1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:12.994047 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1108/2185] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:12 2022] DELETE /identity/v3/projects/0ad35a2fa7f24f46aaf010c78a81ebd2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:13.000330 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b93d1-dcea-41ac-b74e-8408d4be434a None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:13.001298 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b93d1-dcea-41ac-b74e-8408d4be434a None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:13.002920 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b93d1-dcea-41ac-b74e-8408d4be434a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:13.025014 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-d25b93d1-dcea-41ac-b74e-8408d4be434a None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:13.026731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d25b93d1-dcea-41ac-b74e-8408d4be434a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:13.093298 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1078/2186] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:13.101144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fc663-4c00-4a55-82d6-1c3cebe21526 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:13.101429 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fc663-4c00-4a55-82d6-1c3cebe21526 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:13.101707 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fc663-4c00-4a55-82d6-1c3cebe21526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:13.123107 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e03fc663-4c00-4a55-82d6-1c3cebe21526 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:13.125018 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e03fc663-4c00-4a55-82d6-1c3cebe21526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:13.136376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1109/2187] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:13.137467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b93d1-dcea-41ac-b74e-8408d4be434a None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:13.147206 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b93d1-dcea-41ac-b74e-8408d4be434a None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:13.151905 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9ff5e-57ae-4c17-8bbd-745f75451774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:13.152507 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9ff5e-57ae-4c17-8bbd-745f75451774 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:13.152784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9ff5e-57ae-4c17-8bbd-745f75451774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:13.153053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9ff5e-57ae-4c17-8bbd-745f75451774 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:13.153693 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9ff5e-57ae-4c17-8bbd-745f75451774 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:13.154504 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9ff5e-57ae-4c17-8bbd-745f75451774 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:13.155940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c9ff5e-57ae-4c17-8bbd-745f75451774 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:13.167883 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1079/2188] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:34:13.172764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fc663-4c00-4a55-82d6-1c3cebe21526 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:13.182444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fc663-4c00-4a55-82d6-1c3cebe21526 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:13.187640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac8c23-20b7-4909-9c30-a6a0bf9e3818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:13.188236 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac8c23-20b7-4909-9c30-a6a0bf9e3818 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:13.188516 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac8c23-20b7-4909-9c30-a6a0bf9e3818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:13.188785 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac8c23-20b7-4909-9c30-a6a0bf9e3818 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:13.189408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac8c23-20b7-4909-9c30-a6a0bf9e3818 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:13.191549 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac8c23-20b7-4909-9c30-a6a0bf9e3818 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:13.225486 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9ff5e-57ae-4c17-8bbd-745f75451774 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:13.227373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-27c9ff5e-57ae-4c17-8bbd-745f75451774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:13.235658 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1110/2189] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:13 2022] POST /identity/v3/projects => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:13.243731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9ff5e-57ae-4c17-8bbd-745f75451774 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:13.248417 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac8c23-20b7-4909-9c30-a6a0bf9e3818 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:13.324086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:13.340735 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46601669-746f-4791-b0ed-a968810b6b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:13.341535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46601669-746f-4791-b0ed-a968810b6b71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:13.341747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac8c23-20b7-4909-9c30-a6a0bf9e3818 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:13.342009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46601669-746f-4791-b0ed-a968810b6b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:13.346590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46601669-746f-4791-b0ed-a968810b6b71 admin admin] PATH_INFO: `/v3/users/0329ee737db345bfaaa007ffb4c2fdee` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:13.349021 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a0445-933c-4c55-a4db-03ad077778e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:13.349561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a0445-933c-4c55-a4db-03ad077778e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:13.349986 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a0445-933c-4c55-a4db-03ad077778e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:13.350833 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a0445-933c-4c55-a4db-03ad077778e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:13.351480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a0445-933c-4c55-a4db-03ad077778e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:13.355230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46601669-746f-4791-b0ed-a968810b6b71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0329ee737db345bfaaa007ffb4c2fdee)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:13.355650 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a0445-933c-4c55-a4db-03ad077778e2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:13.357479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46601669-746f-4791-b0ed-a968810b6b71 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:13.396921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1111/2190] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:13 2022] POST /identity/v3/users => generated 284 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:34:13.398178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a0445-933c-4c55-a4db-03ad077778e2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:13.413344 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a0445-933c-4c55-a4db-03ad077778e2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:13.419089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c16a1-befb-4994-9af5-c7f55ddb92be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:13.420187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c16a1-befb-4994-9af5-c7f55ddb92be admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:13.420187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c16a1-befb-4994-9af5-c7f55ddb92be admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:13.420411 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45c16a1-befb-4994-9af5-c7f55ddb92be admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:13.422087 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45c16a1-befb-4994-9af5-c7f55ddb92be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:13.423971 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45c16a1-befb-4994-9af5-c7f55ddb92be admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:13.426995 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-46601669-746f-4791-b0ed-a968810b6b71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0329ee737db345bfaaa007ffb4c2fdee'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:13.428026 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-46601669-746f-4791-b0ed-a968810b6b71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0329ee737db345bfaaa007ffb4c2fdee'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:13.428781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-46601669-746f-4791-b0ed-a968810b6b71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0329ee737db345bfaaa007ffb4c2fdee'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:13.439834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-46601669-746f-4791-b0ed-a968810b6b71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0329ee737db345bfaaa007ffb4c2fdee'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:13.441304 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1112/2191] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:13.442640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c16a1-befb-4994-9af5-c7f55ddb92be admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:13.445817 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e45c16a1-befb-4994-9af5-c7f55ddb92be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:13.449214 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-46601669-746f-4791-b0ed-a968810b6b71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0329ee737db345bfaaa007ffb4c2fdee'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:13.474871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1080/2192] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:13 2022] DELETE /identity/v3/users/0329ee737db345bfaaa007ffb4c2fdee => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:13.475221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45c16a1-befb-4994-9af5-c7f55ddb92be admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:13.476633 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbfa7f-6524-4f98-ba0d-3ce2c4018bf3 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:13.476983 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbfa7f-6524-4f98-ba0d-3ce2c4018bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:13.477324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbfa7f-6524-4f98-ba0d-3ce2c4018bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:13.504020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-7abbfa7f-6524-4f98-ba0d-3ce2c4018bf3 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:13.505378 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7abbfa7f-6524-4f98-ba0d-3ce2c4018bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:13.561574 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:13.578256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07bae8-868b-4b92-a515-238d9f3c4db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:13.579110 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07bae8-868b-4b92-a515-238d9f3c4db9 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:13.579456 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07bae8-868b-4b92-a515-238d9f3c4db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:13.579837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07bae8-868b-4b92-a515-238d9f3c4db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:13.580736 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07bae8-868b-4b92-a515-238d9f3c4db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:13.583052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07bae8-868b-4b92-a515-238d9f3c4db9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:13.591424 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1081/2193] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:13.597258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1113/2194] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:13.602454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abbfa7f-6524-4f98-ba0d-3ce2c4018bf3 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:13.602697 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07bae8-868b-4b92-a515-238d9f3c4db9 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:13.605929 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ac07bae8-868b-4b92-a515-238d9f3c4db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:13.614895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abbfa7f-6524-4f98-ba0d-3ce2c4018bf3 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:13.628408 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f720f-ee91-445c-a8a3-868ad76840d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:13.629024 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f720f-ee91-445c-a8a3-868ad76840d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:13.629304 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f720f-ee91-445c-a8a3-868ad76840d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:13.629575 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f720f-ee91-445c-a8a3-868ad76840d6 admin admin] PATH_INFO: `/v3/projects/27aeadc4236647a2aff7f672d4e3c7e8/users/5b65f6e6a765475f97ce9a63ba98f705/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:13.646588 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f720f-ee91-445c-a8a3-868ad76840d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27aeadc4236647a2aff7f672d4e3c7e8, user_id=5b65f6e6a765475f97ce9a63ba98f705, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:13.649595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f720f-ee91-445c-a8a3-868ad76840d6 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:13.665574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1082/2195] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:13 2022] PUT /identity/v3/projects/27aeadc4236647a2aff7f672d4e3c7e8/users/5b65f6e6a765475f97ce9a63ba98f705/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:13.677880 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e4381-1fe6-4efc-be63-2835189e18e8 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:13.678173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e4381-1fe6-4efc-be63-2835189e18e8 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:13.678432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866e4381-1fe6-4efc-be63-2835189e18e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:13.685767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07bae8-868b-4b92-a515-238d9f3c4db9 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:13.718078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-866e4381-1fe6-4efc-be63-2835189e18e8 None None] MFA Rules not processed for user `5b65f6e6a765475f97ce9a63ba98f705`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:13.719733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-866e4381-1fe6-4efc-be63-2835189e18e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:13.755118 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe5ce0-8614-44b2-a2f6-0aef7128351f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:13.755912 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe5ce0-8614-44b2-a2f6-0aef7128351f service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:13.756169 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe5ce0-8614-44b2-a2f6-0aef7128351f service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:13.756442 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe5ce0-8614-44b2-a2f6-0aef7128351f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:13.763674 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:13.809417 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1083/2196] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:13.814755 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e4381-1fe6-4efc-be63-2835189e18e8 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:13.838280 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe5ce0-8614-44b2-a2f6-0aef7128351f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:13.851874 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe5ce0-8614-44b2-a2f6-0aef7128351f service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:13.854916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1114/2197] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:13.920161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866e4381-1fe6-4efc-be63-2835189e18e8 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:13.925911 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7671eb9-bc37-4e9e-b2bb-a7834bdb88ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:13.926586 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7671eb9-bc37-4e9e-b2bb-a7834bdb88ac admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:13.926871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7671eb9-bc37-4e9e-b2bb-a7834bdb88ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:13.927490 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7671eb9-bc37-4e9e-b2bb-a7834bdb88ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:13.927928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7671eb9-bc37-4e9e-b2bb-a7834bdb88ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:13.930134 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7671eb9-bc37-4e9e-b2bb-a7834bdb88ac admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:13.963573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1084/2198] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:13 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:13.969591 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe5ce0-8614-44b2-a2f6-0aef7128351f service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:13.980527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe5ce0-8614-44b2-a2f6-0aef7128351f service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:14.051185 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:14.062281 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8fb5e-69b6-45d4-8752-f0a5b48ea5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:14.062925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8fb5e-69b6-45d4-8752-f0a5b48ea5b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.063208 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8fb5e-69b6-45d4-8752-f0a5b48ea5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.063491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e8fb5e-69b6-45d4-8752-f0a5b48ea5b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.064149 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e8fb5e-69b6-45d4-8752-f0a5b48ea5b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:14.066080 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e8fb5e-69b6-45d4-8752-f0a5b48ea5b8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:14.088336 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1115/2199] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:13 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:14.092300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7671eb9-bc37-4e9e-b2bb-a7834bdb88ac admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:14.103733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7671eb9-bc37-4e9e-b2bb-a7834bdb88ac admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:14.108540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b18863-4ce0-447a-9d8d-a7d2c4e2aec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:14.109145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b18863-4ce0-447a-9d8d-a7d2c4e2aec4 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.109421 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b18863-4ce0-447a-9d8d-a7d2c4e2aec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.109690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b18863-4ce0-447a-9d8d-a7d2c4e2aec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.110334 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b18863-4ce0-447a-9d8d-a7d2c4e2aec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:14.111952 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b18863-4ce0-447a-9d8d-a7d2c4e2aec4 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:14.122003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1085/2200] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:14.127009 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8fb5e-69b6-45d4-8752-f0a5b48ea5b8 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:14.137175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e8fb5e-69b6-45d4-8752-f0a5b48ea5b8 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:14.142575 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e46010-99b7-4a0d-a6d5-5ee2a727eca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:14.143349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e46010-99b7-4a0d-a6d5-5ee2a727eca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.143664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e46010-99b7-4a0d-a6d5-5ee2a727eca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.143949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e46010-99b7-4a0d-a6d5-5ee2a727eca0 admin admin] PATH_INFO: `/v3/projects/42e1cb1e74404cc8a0a6ebac3c9a3b25/users/52bea0d8d831478abafafc84ae7a3bab/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.159924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e46010-99b7-4a0d-a6d5-5ee2a727eca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e1cb1e74404cc8a0a6ebac3c9a3b25, user_id=52bea0d8d831478abafafc84ae7a3bab, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:14.162537 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e46010-99b7-4a0d-a6d5-5ee2a727eca0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:14.177271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1116/2201] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:14 2022] PUT /identity/v3/projects/42e1cb1e74404cc8a0a6ebac3c9a3b25/users/52bea0d8d831478abafafc84ae7a3bab/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:14.181534 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b18863-4ce0-447a-9d8d-a7d2c4e2aec4 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:14.191758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e46010-99b7-4a0d-a6d5-5ee2a727eca0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:14.193850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-72e46010-99b7-4a0d-a6d5-5ee2a727eca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:14.195183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b18863-4ce0-447a-9d8d-a7d2c4e2aec4 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:14.261004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd924c6-32cf-4c30-8873-0af9abf07ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:14.261674 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd924c6-32cf-4c30-8873-0af9abf07ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.262016 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd924c6-32cf-4c30-8873-0af9abf07ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.262309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd924c6-32cf-4c30-8873-0af9abf07ad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.262996 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd924c6-32cf-4c30-8873-0af9abf07ad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:14.265621 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd924c6-32cf-4c30-8873-0af9abf07ad6 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:14.270796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e46010-99b7-4a0d-a6d5-5ee2a727eca0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:14.275562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088adcd-6740-4402-8fc2-31cc54b8c406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:14.275980 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1086/2202] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:14.276258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088adcd-6740-4402-8fc2-31cc54b8c406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.276539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088adcd-6740-4402-8fc2-31cc54b8c406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.276813 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088adcd-6740-4402-8fc2-31cc54b8c406 admin admin] PATH_INFO: `/v3/projects/f3c01cea5def48e39c3b4eefc407123e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.277943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088adcd-6740-4402-8fc2-31cc54b8c406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3c01cea5def48e39c3b4eefc407123e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:14.279928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088adcd-6740-4402-8fc2-31cc54b8c406 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:14.280153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd924c6-32cf-4c30-8873-0af9abf07ad6 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:14.289886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd924c6-32cf-4c30-8873-0af9abf07ad6 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:14.294502 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7935b4b-6780-42ce-adcf-fdd461b9df50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:14.295276 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7935b4b-6780-42ce-adcf-fdd461b9df50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.295432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7935b4b-6780-42ce-adcf-fdd461b9df50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.295760 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7935b4b-6780-42ce-adcf-fdd461b9df50 admin admin] PATH_INFO: `/v3/projects/42e1cb1e74404cc8a0a6ebac3c9a3b25/users/52bea0d8d831478abafafc84ae7a3bab/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.298000 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7935b4b-6780-42ce-adcf-fdd461b9df50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e1cb1e74404cc8a0a6ebac3c9a3b25, user_id=52bea0d8d831478abafafc84ae7a3bab, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:14.300704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7935b4b-6780-42ce-adcf-fdd461b9df50 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:14.313758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1087/2203] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:14 2022] PUT /identity/v3/projects/42e1cb1e74404cc8a0a6ebac3c9a3b25/users/52bea0d8d831478abafafc84ae7a3bab/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:14.324155 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d611a8-7fe2-4b75-b9ba-b353a6544189 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.324442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d611a8-7fe2-4b75-b9ba-b353a6544189 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.324718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d611a8-7fe2-4b75-b9ba-b353a6544189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.333064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1088adcd-6740-4402-8fc2-31cc54b8c406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3c01cea5def48e39c3b4eefc407123e', 'request_id': 'req-1088adcd-6740-4402-8fc2-31cc54b8c406'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:14.338886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1088adcd-6740-4402-8fc2-31cc54b8c406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3c01cea5def48e39c3b4eefc407123e', 'request_id': 'req-1088adcd-6740-4402-8fc2-31cc54b8c406'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:14.348092 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1088adcd-6740-4402-8fc2-31cc54b8c406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3c01cea5def48e39c3b4eefc407123e', 'request_id': 'req-1088adcd-6740-4402-8fc2-31cc54b8c406'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:14.354147 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-b8d611a8-7fe2-4b75-b9ba-b353a6544189 None None] MFA Rules not processed for user `52bea0d8d831478abafafc84ae7a3bab`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:14.355462 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b8d611a8-7fe2-4b75-b9ba-b353a6544189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:14.363756 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1088adcd-6740-4402-8fc2-31cc54b8c406 admin admin] The token cache is being invalidate because project f3c01cea5def48e39c3b4eefc407123e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:14.364318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1088adcd-6740-4402-8fc2-31cc54b8c406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:14.365405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1088adcd-6740-4402-8fc2-31cc54b8c406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:14.368000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1117/2204] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:14 2022] DELETE /identity/v3/projects/f3c01cea5def48e39c3b4eefc407123e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:14.375863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75824c2c-acbd-4bb6-8416-3b7c120169f3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.376150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75824c2c-acbd-4bb6-8416-3b7c120169f3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.376428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75824c2c-acbd-4bb6-8416-3b7c120169f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.383228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:14.397485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-75824c2c-acbd-4bb6-8416-3b7c120169f3 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:14.398804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-75824c2c-acbd-4bb6-8416-3b7c120169f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:14.437346 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1088/2205] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:14.448647 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d611a8-7fe2-4b75-b9ba-b353a6544189 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:14.449322 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:14.450095 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b8d611a8-7fe2-4b75-b9ba-b353a6544189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:14.470229 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1118/2206] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:14.477558 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57f5d8-a283-4625-b143-c1c27126a056 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.477899 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57f5d8-a283-4625-b143-c1c27126a056 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.478176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57f5d8-a283-4625-b143-c1c27126a056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.500175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8b57f5d8-a283-4625-b143-c1c27126a056 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:14.501397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8b57f5d8-a283-4625-b143-c1c27126a056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:14.509312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1119/2207] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:14.512313 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d611a8-7fe2-4b75-b9ba-b353a6544189 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:14.513594 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57f5d8-a283-4625-b143-c1c27126a056 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:14.516477 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bdd460-46fd-419a-8aed-d57f8a28cf56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:14.517140 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdd460-46fd-419a-8aed-d57f8a28cf56 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.517422 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdd460-46fd-419a-8aed-d57f8a28cf56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.517696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdd460-46fd-419a-8aed-d57f8a28cf56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.523762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57f5d8-a283-4625-b143-c1c27126a056 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:14.527975 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bdd460-46fd-419a-8aed-d57f8a28cf56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:14.528486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32916f79-d76c-4b5b-9b79-0579e2a197e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:14.529081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32916f79-d76c-4b5b-9b79-0579e2a197e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.529361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32916f79-d76c-4b5b-9b79-0579e2a197e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.529470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bdd460-46fd-419a-8aed-d57f8a28cf56 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:14.529635 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32916f79-d76c-4b5b-9b79-0579e2a197e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.530635 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32916f79-d76c-4b5b-9b79-0579e2a197e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:14.531012 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32916f79-d76c-4b5b-9b79-0579e2a197e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:14.532639 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32916f79-d76c-4b5b-9b79-0579e2a197e2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:14.540209 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1089/2208] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:14.544779 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1120/2209] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:34:14.549594 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bdd460-46fd-419a-8aed-d57f8a28cf56 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:14.559675 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bdd460-46fd-419a-8aed-d57f8a28cf56 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:14.566244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f29c5a-514b-4225-ad7d-1dda91ed991a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:14.566982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f29c5a-514b-4225-ad7d-1dda91ed991a admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.567269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f29c5a-514b-4225-ad7d-1dda91ed991a admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.567547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f29c5a-514b-4225-ad7d-1dda91ed991a admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.568303 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f29c5a-514b-4225-ad7d-1dda91ed991a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:14.570376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f29c5a-514b-4225-ad7d-1dda91ed991a admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:14.601704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1090/2210] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:14 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:14.603175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32916f79-d76c-4b5b-9b79-0579e2a197e2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:14.614179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32916f79-d76c-4b5b-9b79-0579e2a197e2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:14.628234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f29c5a-514b-4225-ad7d-1dda91ed991a admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:14.675207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7050403-0bf5-4e77-b59a-84271939c3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:14.676178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7050403-0bf5-4e77-b59a-84271939c3c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.676178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7050403-0bf5-4e77-b59a-84271939c3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.676576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7050403-0bf5-4e77-b59a-84271939c3c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.677587 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7050403-0bf5-4e77-b59a-84271939c3c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:14.679384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7050403-0bf5-4e77-b59a-84271939c3c4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:14.685560 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f29c5a-514b-4225-ad7d-1dda91ed991a admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:14.690907 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16395475-0562-4f48-968b-d5896a2df7ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:14.691929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16395475-0562-4f48-968b-d5896a2df7ba service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.692304 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16395475-0562-4f48-968b-d5896a2df7ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.692603 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16395475-0562-4f48-968b-d5896a2df7ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.694387 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-16395475-0562-4f48-968b-d5896a2df7ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:14.703731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1121/2211] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:14 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:14.707879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7050403-0bf5-4e77-b59a-84271939c3c4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:14.719951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7050403-0bf5-4e77-b59a-84271939c3c4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:14.726244 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c5422-58fb-4ea7-810c-1c5a11c8d5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:14.726862 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c5422-58fb-4ea7-810c-1c5a11c8d5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.727146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c5422-58fb-4ea7-810c-1c5a11c8d5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.727425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c5422-58fb-4ea7-810c-1c5a11c8d5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.730086 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c5422-58fb-4ea7-810c-1c5a11c8d5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:14.730086 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c5422-58fb-4ea7-810c-1c5a11c8d5e1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:14.741916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1122/2212] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:14.748172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c5422-58fb-4ea7-810c-1c5a11c8d5e1 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:14.761286 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c5422-58fb-4ea7-810c-1c5a11c8d5e1 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:14.769710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:14.771336 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af13a3-c78d-4553-8840-0d73381258d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:14.772065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af13a3-c78d-4553-8840-0d73381258d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.772352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af13a3-c78d-4553-8840-0d73381258d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.772624 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57af13a3-c78d-4553-8840-0d73381258d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.773366 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af13a3-c78d-4553-8840-0d73381258d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:14.774404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16395475-0562-4f48-968b-d5896a2df7ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:14.774892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57af13a3-c78d-4553-8840-0d73381258d6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:14.776546 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16395475-0562-4f48-968b-d5896a2df7ba service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:14.787225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1123/2213] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:34:14.788832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1091/2214] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:34:14.790924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af13a3-c78d-4553-8840-0d73381258d6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:14.802723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57af13a3-c78d-4553-8840-0d73381258d6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:14.807933 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60026c23-4181-4a21-8318-d9871046bd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:14.808652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60026c23-4181-4a21-8318-d9871046bd84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.809002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60026c23-4181-4a21-8318-d9871046bd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.809350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60026c23-4181-4a21-8318-d9871046bd84 admin admin] PATH_INFO: `/v3/projects/aee96e55a6a34dbb81847961d87abad0/users/18f8f419b523440f8a4ca06dc83bff40/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.825135 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60026c23-4181-4a21-8318-d9871046bd84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aee96e55a6a34dbb81847961d87abad0, user_id=18f8f419b523440f8a4ca06dc83bff40, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:14.828932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60026c23-4181-4a21-8318-d9871046bd84 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:14.843754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1124/2215] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:14 2022] PUT /identity/v3/projects/aee96e55a6a34dbb81847961d87abad0/users/18f8f419b523440f8a4ca06dc83bff40/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:14.849302 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ceeaa-523c-447a-8dc5-d3074f289adc None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:14.849593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ceeaa-523c-447a-8dc5-d3074f289adc None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:14.849891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ceeaa-523c-447a-8dc5-d3074f289adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:14.882748 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-063ceeaa-523c-447a-8dc5-d3074f289adc None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:14.884008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-063ceeaa-523c-447a-8dc5-d3074f289adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:14.973405 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1092/2216] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:14.982918 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60026c23-4181-4a21-8318-d9871046bd84 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:14.985643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-60026c23-4181-4a21-8318-d9871046bd84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:15.089089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60026c23-4181-4a21-8318-d9871046bd84 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:15.094856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e917c1-7699-4e2e-8d93-0b75f1883f5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:15.095755 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e917c1-7699-4e2e-8d93-0b75f1883f5c service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:15.096082 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e917c1-7699-4e2e-8d93-0b75f1883f5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:15.096364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e917c1-7699-4e2e-8d93-0b75f1883f5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:15.117023 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e917c1-7699-4e2e-8d93-0b75f1883f5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:15.120544 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e917c1-7699-4e2e-8d93-0b75f1883f5c service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:15.148722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1125/2217] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:18.637446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ebb33-95d7-48e2-91e9-b06918b31ea5 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:18.638059 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ebb33-95d7-48e2-91e9-b06918b31ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:18.638059 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ebb33-95d7-48e2-91e9-b06918b31ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:18.662299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-cc8ebb33-95d7-48e2-91e9-b06918b31ea5 None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:18.664272 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-cc8ebb33-95d7-48e2-91e9-b06918b31ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:18.742387 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1093/2218] 10.222.0.58 () {58 vars in 1009 bytes} [Wed Aug 24 12:34:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:18.801955 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e917c1-7699-4e2e-8d93-0b75f1883f5c service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:18.803525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-44e917c1-7699-4e2e-8d93-0b75f1883f5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:18.824445 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:18.905515 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e917c1-7699-4e2e-8d93-0b75f1883f5c service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:18.913088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84a788-e510-414a-be32-94f801a20985 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:18.913927 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84a788-e510-414a-be32-94f801a20985 service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:18.914322 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84a788-e510-414a-be32-94f801a20985 service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:18.914645 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a84a788-e510-414a-be32-94f801a20985 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:18.925958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84a788-e510-414a-be32-94f801a20985 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:18.927678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a84a788-e510-414a-be32-94f801a20985 service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:18.942439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1126/2219] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:34:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:34:19.758224 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f157e8-067a-4b8a-8cbc-0bd62bc1876c None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:19.758702 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f157e8-067a-4b8a-8cbc-0bd62bc1876c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:19.758811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f157e8-067a-4b8a-8cbc-0bd62bc1876c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:19.784292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-08f157e8-067a-4b8a-8cbc-0bd62bc1876c None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:19.785996 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-08f157e8-067a-4b8a-8cbc-0bd62bc1876c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:19.794891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1094/2220] 10.222.0.58 () {58 vars in 967 bytes} [Wed Aug 24 12:34:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:19.809140 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84a788-e510-414a-be32-94f801a20985 service swift] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:19.822538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a84a788-e510-414a-be32-94f801a20985 service swift] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:19.831194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36fe28-1429-46a0-8490-17b754e0b9fa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:19.831982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36fe28-1429-46a0-8490-17b754e0b9fa service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:19.832330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36fe28-1429-46a0-8490-17b754e0b9fa service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:19.832729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36fe28-1429-46a0-8490-17b754e0b9fa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:19.853307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36fe28-1429-46a0-8490-17b754e0b9fa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:19.855457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36fe28-1429-46a0-8490-17b754e0b9fa service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:19.866698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1127/2221] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:34:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:34:20.227241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909939e8-f405-48e1-a7bf-5bc408b6ef91 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:20.227892 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909939e8-f405-48e1-a7bf-5bc408b6ef91 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:20.228334 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909939e8-f405-48e1-a7bf-5bc408b6ef91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:20.253150 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-909939e8-f405-48e1-a7bf-5bc408b6ef91 None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:20.255092 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-909939e8-f405-48e1-a7bf-5bc408b6ef91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:20.267252 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1095/2222] 10.222.0.58 () {58 vars in 1009 bytes} [Wed Aug 24 12:34:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:20.280062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36fe28-1429-46a0-8490-17b754e0b9fa service swift] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:20.294871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36fe28-1429-46a0-8490-17b754e0b9fa service swift] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:20.301078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37ee09-1647-46de-a397-f64777fa170a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:20.302090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37ee09-1647-46de-a397-f64777fa170a service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:20.303300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37ee09-1647-46de-a397-f64777fa170a service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:20.303807 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab37ee09-1647-46de-a397-f64777fa170a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:20.315522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37ee09-1647-46de-a397-f64777fa170a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:20.317345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab37ee09-1647-46de-a397-f64777fa170a service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:20.328801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1128/2223] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:34:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:34:20.798924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda3eec-5190-476f-86bf-8cbd9eafac50 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:20.799416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda3eec-5190-476f-86bf-8cbd9eafac50 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:20.799775 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda3eec-5190-476f-86bf-8cbd9eafac50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:20.825664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-efda3eec-5190-476f-86bf-8cbd9eafac50 None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:20.826960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-efda3eec-5190-476f-86bf-8cbd9eafac50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:20.850014 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1096/2224] 10.222.0.58 () {58 vars in 967 bytes} [Wed Aug 24 12:34:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:20.866784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37ee09-1647-46de-a397-f64777fa170a service swift] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:20.880011 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab37ee09-1647-46de-a397-f64777fa170a service swift] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:20.886372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036eeab-5cbe-44a8-b6b7-981f35b6c4da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:20.887218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036eeab-5cbe-44a8-b6b7-981f35b6c4da service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:20.887669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036eeab-5cbe-44a8-b6b7-981f35b6c4da service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:20.888267 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036eeab-5cbe-44a8-b6b7-981f35b6c4da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:20.906302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036eeab-5cbe-44a8-b6b7-981f35b6c4da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:20.908891 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5036eeab-5cbe-44a8-b6b7-981f35b6c4da service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:20.912936 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66788e2-712c-42ec-9f10-972a064d2479 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:20.913229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66788e2-712c-42ec-9f10-972a064d2479 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:20.913513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66788e2-712c-42ec-9f10-972a064d2479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:20.923605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1129/2225] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:34:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:34:20.959189 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-e66788e2-712c-42ec-9f10-972a064d2479 None None] MFA Rules not processed for user `18f8f419b523440f8a4ca06dc83bff40`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:20.960661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e66788e2-712c-42ec-9f10-972a064d2479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:21.037409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1097/2226] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:21.042426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036eeab-5cbe-44a8-b6b7-981f35b6c4da service swift] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:21.059615 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036eeab-5cbe-44a8-b6b7-981f35b6c4da service swift] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:21.087859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80596890-448d-47b7-bd97-97e7fc5e484f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:21.088835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80596890-448d-47b7-bd97-97e7fc5e484f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:21.089111 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80596890-448d-47b7-bd97-97e7fc5e484f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:21.095113 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80596890-448d-47b7-bd97-97e7fc5e484f admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:21.096107 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80596890-448d-47b7-bd97-97e7fc5e484f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:21.098054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80596890-448d-47b7-bd97-97e7fc5e484f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:21.137360 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1130/2227] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:21 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:21.141943 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66788e2-712c-42ec-9f10-972a064d2479 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:21.153306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66788e2-712c-42ec-9f10-972a064d2479 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:21.163614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:21.252138 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82fa7f4-2886-4857-bccb-8483ede603c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:21.252886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fa7f4-2886-4857-bccb-8483ede603c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:21.253237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fa7f4-2886-4857-bccb-8483ede603c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:21.253512 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fa7f4-2886-4857-bccb-8483ede603c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:21.254202 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82fa7f4-2886-4857-bccb-8483ede603c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:21.256347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82fa7f4-2886-4857-bccb-8483ede603c4 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:21.284871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1098/2228] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:21 2022] POST /identity/v3/users => generated 295 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:21.288891 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80596890-448d-47b7-bd97-97e7fc5e484f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:21.307986 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80596890-448d-47b7-bd97-97e7fc5e484f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:21.315899 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c84acf-1a95-4238-9b71-65b780163a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:21.316721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c84acf-1a95-4238-9b71-65b780163a44 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:21.317008 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c84acf-1a95-4238-9b71-65b780163a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:21.317277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c84acf-1a95-4238-9b71-65b780163a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:21.318093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c84acf-1a95-4238-9b71-65b780163a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:21.319723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c84acf-1a95-4238-9b71-65b780163a44 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:21.332741 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1131/2229] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:21.337353 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82fa7f4-2886-4857-bccb-8483ede603c4 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:21.372898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82fa7f4-2886-4857-bccb-8483ede603c4 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:21.385035 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b244e3-553e-4a79-88d0-1b2af8d9fb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:21.385966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b244e3-553e-4a79-88d0-1b2af8d9fb6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:21.386221 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b244e3-553e-4a79-88d0-1b2af8d9fb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:21.386366 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b244e3-553e-4a79-88d0-1b2af8d9fb6a admin admin] PATH_INFO: `/v3/projects/159cb10e789d47ba9c4f865c05e07491/users/e3df898326404ea4b1343fbcb5d70a9d/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:21.407929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b244e3-553e-4a79-88d0-1b2af8d9fb6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=159cb10e789d47ba9c4f865c05e07491, user_id=e3df898326404ea4b1343fbcb5d70a9d, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:21.413853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b244e3-553e-4a79-88d0-1b2af8d9fb6a admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:21.447928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1099/2230] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:21 2022] PUT /identity/v3/projects/159cb10e789d47ba9c4f865c05e07491/users/e3df898326404ea4b1343fbcb5d70a9d/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:21.452240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c84acf-1a95-4238-9b71-65b780163a44 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:21.467093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c84acf-1a95-4238-9b71-65b780163a44 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:21.598819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d05c9-9b0b-452d-8755-4b255223e077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:21.601736 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d05c9-9b0b-452d-8755-4b255223e077 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:21.602048 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d05c9-9b0b-452d-8755-4b255223e077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:21.602540 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d05c9-9b0b-452d-8755-4b255223e077 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:21.603465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d05c9-9b0b-452d-8755-4b255223e077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:21.605312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d05c9-9b0b-452d-8755-4b255223e077 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:21.621567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1132/2231] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:21 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:34:21.625965 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b244e3-553e-4a79-88d0-1b2af8d9fb6a admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:21.644162 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b244e3-553e-4a79-88d0-1b2af8d9fb6a admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:21.651088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b1103-8805-499f-a251-b798c272f659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:21.652325 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b1103-8805-499f-a251-b798c272f659 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:21.652611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b1103-8805-499f-a251-b798c272f659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:21.652880 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b1103-8805-499f-a251-b798c272f659 admin admin] PATH_INFO: `/v3/projects/159cb10e789d47ba9c4f865c05e07491/users/e3df898326404ea4b1343fbcb5d70a9d/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:21.656771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b1103-8805-499f-a251-b798c272f659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=159cb10e789d47ba9c4f865c05e07491, user_id=e3df898326404ea4b1343fbcb5d70a9d, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:21.659139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b1103-8805-499f-a251-b798c272f659 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:21.660837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d05c9-9b0b-452d-8755-4b255223e077 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:21.662659 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-851d05c9-9b0b-452d-8755-4b255223e077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:21.685342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d05c9-9b0b-452d-8755-4b255223e077 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:21.723962 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c0471-bb57-400f-967c-6767ad5ecaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:21.724135 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0471-bb57-400f-967c-6767ad5ecaeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:21.724403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0471-bb57-400f-967c-6767ad5ecaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:21.724825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0471-bb57-400f-967c-6767ad5ecaeb admin admin] PATH_INFO: `/v3/users/5b65f6e6a765475f97ce9a63ba98f705` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:21.730742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c0471-bb57-400f-967c-6767ad5ecaeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b65f6e6a765475f97ce9a63ba98f705)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:21.738548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1100/2232] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:21 2022] PUT /identity/v3/projects/159cb10e789d47ba9c4f865c05e07491/users/e3df898326404ea4b1343fbcb5d70a9d/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:21.752691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c0471-bb57-400f-967c-6767ad5ecaeb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:21.835953 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:21.872549 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ab7c0471-bb57-400f-967c-6767ad5ecaeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b65f6e6a765475f97ce9a63ba98f705'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.873498 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ab7c0471-bb57-400f-967c-6767ad5ecaeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b65f6e6a765475f97ce9a63ba98f705'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.884824 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ab7c0471-bb57-400f-967c-6767ad5ecaeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b65f6e6a765475f97ce9a63ba98f705'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.901143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ab7c0471-bb57-400f-967c-6767ad5ecaeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b65f6e6a765475f97ce9a63ba98f705'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.945989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ab7c0471-bb57-400f-967c-6767ad5ecaeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b65f6e6a765475f97ce9a63ba98f705'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:21.972948 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1133/2233] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:21 2022] DELETE /identity/v3/users/5b65f6e6a765475f97ce9a63ba98f705 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:21.978710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd850d-c28a-4855-9630-aae51a7de007 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:21.980053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd850d-c28a-4855-9630-aae51a7de007 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:21.980336 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd850d-c28a-4855-9630-aae51a7de007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:22.020823 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-86fd850d-c28a-4855-9630-aae51a7de007 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:22.022398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-86fd850d-c28a-4855-9630-aae51a7de007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:22.126619 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1101/2234] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:22.135283 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c0471-bb57-400f-967c-6767ad5ecaeb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:22.136797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ab7c0471-bb57-400f-967c-6767ad5ecaeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:22.283319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c0471-bb57-400f-967c-6767ad5ecaeb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:22.293229 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6228f7f-cb60-4aeb-8e6d-82d2e7566791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:22.295267 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6228f7f-cb60-4aeb-8e6d-82d2e7566791 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:22.295949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6228f7f-cb60-4aeb-8e6d-82d2e7566791 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:22.297110 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6228f7f-cb60-4aeb-8e6d-82d2e7566791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:22.322739 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6228f7f-cb60-4aeb-8e6d-82d2e7566791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:22.324395 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6228f7f-cb60-4aeb-8e6d-82d2e7566791 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:22.352775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1134/2235] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:22.778029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd850d-c28a-4855-9630-aae51a7de007 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:22.780284 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-86fd850d-c28a-4855-9630-aae51a7de007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:22.796933 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd850d-c28a-4855-9630-aae51a7de007 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:22.815124 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e3469-05e5-411f-ad67-019e08cfde6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:22.815914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e3469-05e5-411f-ad67-019e08cfde6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:22.816400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e3469-05e5-411f-ad67-019e08cfde6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:22.816871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61e3469-05e5-411f-ad67-019e08cfde6a admin admin] PATH_INFO: `/v3/projects/27aeadc4236647a2aff7f672d4e3c7e8` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:22.818565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e3469-05e5-411f-ad67-019e08cfde6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=27aeadc4236647a2aff7f672d4e3c7e8)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:22.820825 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61e3469-05e5-411f-ad67-019e08cfde6a admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:22.913965 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c61e3469-05e5-411f-ad67-019e08cfde6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27aeadc4236647a2aff7f672d4e3c7e8', 'request_id': 'req-c61e3469-05e5-411f-ad67-019e08cfde6a'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:22.919422 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:22.922370 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c61e3469-05e5-411f-ad67-019e08cfde6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27aeadc4236647a2aff7f672d4e3c7e8', 'request_id': 'req-c61e3469-05e5-411f-ad67-019e08cfde6a'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:22.937528 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c61e3469-05e5-411f-ad67-019e08cfde6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27aeadc4236647a2aff7f672d4e3c7e8', 'request_id': 'req-c61e3469-05e5-411f-ad67-019e08cfde6a'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:22.960229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c61e3469-05e5-411f-ad67-019e08cfde6a admin admin] The token cache is being invalidate because project 27aeadc4236647a2aff7f672d4e3c7e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:22.960487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c61e3469-05e5-411f-ad67-019e08cfde6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:22.961520 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c61e3469-05e5-411f-ad67-019e08cfde6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:22.970178 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1102/2236] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:22 2022] DELETE /identity/v3/projects/27aeadc4236647a2aff7f672d4e3c7e8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:22.970942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6228f7f-cb60-4aeb-8e6d-82d2e7566791 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:22.986386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6228f7f-cb60-4aeb-8e6d-82d2e7566791 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:23.058957 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41ccd3-353d-4de7-8212-34adb000b24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:23.059603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41ccd3-353d-4de7-8212-34adb000b24d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:23.059940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41ccd3-353d-4de7-8212-34adb000b24d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:23.060206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41ccd3-353d-4de7-8212-34adb000b24d admin admin] PATH_INFO: `/v3/users/52bea0d8d831478abafafc84ae7a3bab` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:23.062359 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41ccd3-353d-4de7-8212-34adb000b24d admin admin] RBAC: Authorizing `identity:delete_user(user_id=52bea0d8d831478abafafc84ae7a3bab)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:23.064424 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41ccd3-353d-4de7-8212-34adb000b24d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:23.121765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:23.125064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1d41ccd3-353d-4de7-8212-34adb000b24d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52bea0d8d831478abafafc84ae7a3bab'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:23.126602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1d41ccd3-353d-4de7-8212-34adb000b24d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52bea0d8d831478abafafc84ae7a3bab'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:23.138105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1d41ccd3-353d-4de7-8212-34adb000b24d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52bea0d8d831478abafafc84ae7a3bab'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:23.149601 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1d41ccd3-353d-4de7-8212-34adb000b24d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52bea0d8d831478abafafc84ae7a3bab'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:23.171949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1d41ccd3-353d-4de7-8212-34adb000b24d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52bea0d8d831478abafafc84ae7a3bab'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:23.182298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1135/2237] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:22 2022] DELETE /identity/v3/users/52bea0d8d831478abafafc84ae7a3bab => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:23.766253 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e3469-05e5-411f-ad67-019e08cfde6a admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:23.773188 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c61e3469-05e5-411f-ad67-019e08cfde6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:23.801814 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61e3469-05e5-411f-ad67-019e08cfde6a admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:23.932605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efa136-89c7-4ae3-8448-dde32d2245a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:23.933560 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efa136-89c7-4ae3-8448-dde32d2245a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:23.934201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efa136-89c7-4ae3-8448-dde32d2245a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:23.935289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efa136-89c7-4ae3-8448-dde32d2245a4 admin admin] PATH_INFO: `/v3/projects/42e1cb1e74404cc8a0a6ebac3c9a3b25` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:23.937935 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2efa136-89c7-4ae3-8448-dde32d2245a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e1cb1e74404cc8a0a6ebac3c9a3b25)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:23.941550 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2efa136-89c7-4ae3-8448-dde32d2245a4 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:24.033476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41ccd3-353d-4de7-8212-34adb000b24d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:24.034789 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1d41ccd3-353d-4de7-8212-34adb000b24d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:24.052351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41ccd3-353d-4de7-8212-34adb000b24d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:24.058947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f2efa136-89c7-4ae3-8448-dde32d2245a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e1cb1e74404cc8a0a6ebac3c9a3b25', 'request_id': 'req-f2efa136-89c7-4ae3-8448-dde32d2245a4'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:24.073887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f2efa136-89c7-4ae3-8448-dde32d2245a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e1cb1e74404cc8a0a6ebac3c9a3b25', 'request_id': 'req-f2efa136-89c7-4ae3-8448-dde32d2245a4'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:24.097390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f2efa136-89c7-4ae3-8448-dde32d2245a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e1cb1e74404cc8a0a6ebac3c9a3b25', 'request_id': 'req-f2efa136-89c7-4ae3-8448-dde32d2245a4'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:24.124026 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f2efa136-89c7-4ae3-8448-dde32d2245a4 admin admin] The token cache is being invalidate because project 42e1cb1e74404cc8a0a6ebac3c9a3b25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:24.124309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f2efa136-89c7-4ae3-8448-dde32d2245a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:24.125684 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-f2efa136-89c7-4ae3-8448-dde32d2245a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:24.127202 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1103/2238] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:23 2022] DELETE /identity/v3/projects/42e1cb1e74404cc8a0a6ebac3c9a3b25 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:24.132975 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdde78-a888-421f-8aa0-c7c7ccab72de None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:24.133522 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdde78-a888-421f-8aa0-c7c7ccab72de None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:24.133996 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cdde78-a888-421f-8aa0-c7c7ccab72de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:24.162872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-18cdde78-a888-421f-8aa0-c7c7ccab72de None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:24.164196 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-18cdde78-a888-421f-8aa0-c7c7ccab72de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:24.174503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1104/2239] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:24.182589 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868d7b3-5a79-4352-b063-a1e0ae798ca8 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:24.182974 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868d7b3-5a79-4352-b063-a1e0ae798ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:24.183347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868d7b3-5a79-4352-b063-a1e0ae798ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:24.191054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b7863-814a-4347-832c-5831766e8081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:24.191766 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b7863-814a-4347-832c-5831766e8081 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:24.192056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b7863-814a-4347-832c-5831766e8081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:24.192335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b7863-814a-4347-832c-5831766e8081 admin admin] PATH_INFO: `/v3/users/2af58e3ca6394bfa8ca34007c4ece01a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:24.193823 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b7863-814a-4347-832c-5831766e8081 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2af58e3ca6394bfa8ca34007c4ece01a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:24.196265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b7863-814a-4347-832c-5831766e8081 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:24.251957 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-9868d7b3-5a79-4352-b063-a1e0ae798ca8 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:24.253564 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-9868d7b3-5a79-4352-b063-a1e0ae798ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:24.279181 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1105/2240] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:24.283755 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868d7b3-5a79-4352-b063-a1e0ae798ca8 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:24.300443 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868d7b3-5a79-4352-b063-a1e0ae798ca8 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:24.306276 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1914165-8420-4089-99ae-e3e2ba130246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:24.306991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1914165-8420-4089-99ae-e3e2ba130246 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:24.307350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1914165-8420-4089-99ae-e3e2ba130246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:24.307546 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1914165-8420-4089-99ae-e3e2ba130246 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:24.308435 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1914165-8420-4089-99ae-e3e2ba130246 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:24.308919 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1914165-8420-4089-99ae-e3e2ba130246 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:24.311410 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1914165-8420-4089-99ae-e3e2ba130246 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:24.321383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a86b7863-814a-4347-832c-5831766e8081 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2af58e3ca6394bfa8ca34007c4ece01a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:24.325425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a86b7863-814a-4347-832c-5831766e8081 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2af58e3ca6394bfa8ca34007c4ece01a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:24.326388 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1106/2241] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:34:24.331288 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1914165-8420-4089-99ae-e3e2ba130246 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:24.343567 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1914165-8420-4089-99ae-e3e2ba130246 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:24.345307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a86b7863-814a-4347-832c-5831766e8081 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2af58e3ca6394bfa8ca34007c4ece01a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:24.349312 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308d0cf-467e-4f4f-a89f-cb33453217aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:24.349887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308d0cf-467e-4f4f-a89f-cb33453217aa admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:24.350177 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308d0cf-467e-4f4f-a89f-cb33453217aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:24.350645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e308d0cf-467e-4f4f-a89f-cb33453217aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:24.351365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308d0cf-467e-4f4f-a89f-cb33453217aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:24.353564 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e308d0cf-467e-4f4f-a89f-cb33453217aa admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:24.376259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a86b7863-814a-4347-832c-5831766e8081 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2af58e3ca6394bfa8ca34007c4ece01a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:24.396824 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1107/2242] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:24 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:24.401078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308d0cf-467e-4f4f-a89f-cb33453217aa admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:24.404538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-a86b7863-814a-4347-832c-5831766e8081 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2af58e3ca6394bfa8ca34007c4ece01a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:24.418597 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1136/2243] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:24 2022] DELETE /identity/v3/users/2af58e3ca6394bfa8ca34007c4ece01a => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:24.424233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da2ef7-114d-423b-9392-0f8b845cb808 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:24.424531 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da2ef7-114d-423b-9392-0f8b845cb808 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:24.424814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75da2ef7-114d-423b-9392-0f8b845cb808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:24.446243 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:24.467374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-75da2ef7-114d-423b-9392-0f8b845cb808 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:24.469103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-75da2ef7-114d-423b-9392-0f8b845cb808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:24.526552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:24.528156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e308d0cf-467e-4f4f-a89f-cb33453217aa admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:24.568094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1137/2244] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:24.569142 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da2ef7-114d-423b-9392-0f8b845cb808 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:24.570665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-75da2ef7-114d-423b-9392-0f8b845cb808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:24.681398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dc42b-eaca-4217-8b55-91c73576fb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:24.682093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dc42b-eaca-4217-8b55-91c73576fb70 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:24.682446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dc42b-eaca-4217-8b55-91c73576fb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:24.682794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dc42b-eaca-4217-8b55-91c73576fb70 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:24.683509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dc42b-eaca-4217-8b55-91c73576fb70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:24.685601 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dc42b-eaca-4217-8b55-91c73576fb70 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:24.706085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75da2ef7-114d-423b-9392-0f8b845cb808 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:24.720698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9019e-6b70-45a1-ad92-8da15518fc37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:24.725299 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9019e-6b70-45a1-ad92-8da15518fc37 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:24.725586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9019e-6b70-45a1-ad92-8da15518fc37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:24.725861 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9019e-6b70-45a1-ad92-8da15518fc37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:24.731882 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1108/2245] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:24 2022] POST /identity/v3/users => generated 293 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:24.738804 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dc42b-eaca-4217-8b55-91c73576fb70 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:24.741873 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c64dc42b-eaca-4217-8b55-91c73576fb70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:24.765418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9019e-6b70-45a1-ad92-8da15518fc37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:24.767179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9019e-6b70-45a1-ad92-8da15518fc37 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:24.776923 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dc42b-eaca-4217-8b55-91c73576fb70 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:24.784541 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679334f-5437-44b4-9b74-11eaf6c3b28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:24.785237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679334f-5437-44b4-9b74-11eaf6c3b28e admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:24.785775 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679334f-5437-44b4-9b74-11eaf6c3b28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:24.786114 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679334f-5437-44b4-9b74-11eaf6c3b28e admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:24.787022 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679334f-5437-44b4-9b74-11eaf6c3b28e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:24.788863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1138/2246] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:24.797673 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679334f-5437-44b4-9b74-11eaf6c3b28e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:24.827578 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1109/2247] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:24.828856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9019e-6b70-45a1-ad92-8da15518fc37 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:24.844538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9019e-6b70-45a1-ad92-8da15518fc37 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:24.853944 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5fdf1-a34a-499a-aaf3-f712bd150525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:24.854608 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5fdf1-a34a-499a-aaf3-f712bd150525 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:24.854909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5fdf1-a34a-499a-aaf3-f712bd150525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:24.855126 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5fdf1-a34a-499a-aaf3-f712bd150525 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:24.860338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5fdf1-a34a-499a-aaf3-f712bd150525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:24.862116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f5fdf1-a34a-499a-aaf3-f712bd150525 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:24.885102 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1139/2248] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:24.889489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679334f-5437-44b4-9b74-11eaf6c3b28e admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:24.907131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679334f-5437-44b4-9b74-11eaf6c3b28e admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:24.913001 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a51db-2f4d-4203-b5d6-d51b8a1ffc00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:24.914612 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a51db-2f4d-4203-b5d6-d51b8a1ffc00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:24.915107 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a51db-2f4d-4203-b5d6-d51b8a1ffc00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:24.916208 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a51db-2f4d-4203-b5d6-d51b8a1ffc00 admin admin] PATH_INFO: `/v3/projects/a4b96e33b1f64002bc1b75683bdab2e1/users/4f8b1706b3784751a230ea647de955b8/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:24.945903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a51db-2f4d-4203-b5d6-d51b8a1ffc00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b96e33b1f64002bc1b75683bdab2e1, user_id=4f8b1706b3784751a230ea647de955b8, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:24.950513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a51db-2f4d-4203-b5d6-d51b8a1ffc00 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:24.978706 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1110/2249] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:24 2022] PUT /identity/v3/projects/a4b96e33b1f64002bc1b75683bdab2e1/users/4f8b1706b3784751a230ea647de955b8/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:24.989517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16265d-cd04-4f4f-9986-5fa0912dd6e1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:24.989744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16265d-cd04-4f4f-9986-5fa0912dd6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:24.990073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16265d-cd04-4f4f-9986-5fa0912dd6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:25.036760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-0e16265d-cd04-4f4f-9986-5fa0912dd6e1 None None] MFA Rules not processed for user `4f8b1706b3784751a230ea647de955b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:25.038180 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0e16265d-cd04-4f4f-9986-5fa0912dd6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:25.156212 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1140/2250] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:25.161038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a51db-2f4d-4203-b5d6-d51b8a1ffc00 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:25.245521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a51db-2f4d-4203-b5d6-d51b8a1ffc00 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:25.259483 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c52ea-c655-4133-9d61-5dac8c1f5491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:25.260328 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c52ea-c655-4133-9d61-5dac8c1f5491 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:25.260842 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c52ea-c655-4133-9d61-5dac8c1f5491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:25.261229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c52ea-c655-4133-9d61-5dac8c1f5491 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:25.261955 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c52ea-c655-4133-9d61-5dac8c1f5491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:25.264655 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c52ea-c655-4133-9d61-5dac8c1f5491 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:25.287021 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16265d-cd04-4f4f-9986-5fa0912dd6e1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:25.289076 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0e16265d-cd04-4f4f-9986-5fa0912dd6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:25.320521 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16265d-cd04-4f4f-9986-5fa0912dd6e1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:25.330815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27d48b-1a37-4861-a213-88d1e468a11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:25.332074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27d48b-1a37-4861-a213-88d1e468a11e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:25.332778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27d48b-1a37-4861-a213-88d1e468a11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:25.333374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27d48b-1a37-4861-a213-88d1e468a11e admin admin] PATH_INFO: `/v3/projects/2bdabfd06f064b05a6d8ec06b4fca329` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:25.335323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27d48b-1a37-4861-a213-88d1e468a11e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bdabfd06f064b05a6d8ec06b4fca329)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:25.338146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27d48b-1a37-4861-a213-88d1e468a11e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:25.356461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1111/2251] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:25 2022] POST /identity/v3/projects => generated 387 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:25.357438 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c52ea-c655-4133-9d61-5dac8c1f5491 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:25.396136 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:25.428210 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-af27d48b-1a37-4861-a213-88d1e468a11e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bdabfd06f064b05a6d8ec06b4fca329', 'request_id': 'req-af27d48b-1a37-4861-a213-88d1e468a11e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:25.460530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-af27d48b-1a37-4861-a213-88d1e468a11e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bdabfd06f064b05a6d8ec06b4fca329', 'request_id': 'req-af27d48b-1a37-4861-a213-88d1e468a11e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:25.464888 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:25.485666 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-af27d48b-1a37-4861-a213-88d1e468a11e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bdabfd06f064b05a6d8ec06b4fca329', 'request_id': 'req-af27d48b-1a37-4861-a213-88d1e468a11e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:25.504758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-af27d48b-1a37-4861-a213-88d1e468a11e admin admin] The token cache is being invalidate because project 2bdabfd06f064b05a6d8ec06b4fca329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:25.505272 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-af27d48b-1a37-4861-a213-88d1e468a11e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:25.506778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-af27d48b-1a37-4861-a213-88d1e468a11e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:25.509391 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1141/2252] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:25 2022] DELETE /identity/v3/projects/2bdabfd06f064b05a6d8ec06b4fca329 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:25.518104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c52ea-c655-4133-9d61-5dac8c1f5491 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:25.636142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25a478-b481-4a29-b3b6-8cfc2aedeb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:25.636142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25a478-b481-4a29-b3b6-8cfc2aedeb15 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:25.636142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25a478-b481-4a29-b3b6-8cfc2aedeb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:25.636142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25a478-b481-4a29-b3b6-8cfc2aedeb15 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:25.636768 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25a478-b481-4a29-b3b6-8cfc2aedeb15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:25.636768 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25a478-b481-4a29-b3b6-8cfc2aedeb15 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:25.710561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1112/2253] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:25 2022] POST /identity/v3/users => generated 300 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:25.714542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27d48b-1a37-4861-a213-88d1e468a11e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:25.718609 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-af27d48b-1a37-4861-a213-88d1e468a11e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:25.753309 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27d48b-1a37-4861-a213-88d1e468a11e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:25.766091 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f5b58-5567-41bf-af7d-5c6a0bb24397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:25.766643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f5b58-5567-41bf-af7d-5c6a0bb24397 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:25.766927 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f5b58-5567-41bf-af7d-5c6a0bb24397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:25.767473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f5b58-5567-41bf-af7d-5c6a0bb24397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:25.768178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f5b58-5567-41bf-af7d-5c6a0bb24397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:25.769808 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f5b58-5567-41bf-af7d-5c6a0bb24397 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:25.786652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1142/2254] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:25.790974 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25a478-b481-4a29-b3b6-8cfc2aedeb15 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:25.806565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25a478-b481-4a29-b3b6-8cfc2aedeb15 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:25.830900 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f6312-5689-4c3e-8f6e-a909628e5d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:25.832269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f6312-5689-4c3e-8f6e-a909628e5d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:25.832565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f6312-5689-4c3e-8f6e-a909628e5d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:25.833852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f6312-5689-4c3e-8f6e-a909628e5d35 admin admin] PATH_INFO: `/v3/projects/fedef4141151438283257c52844e4b7e/users/9a9c199b160444c2a4619c742a00ac31/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:25.852433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f6312-5689-4c3e-8f6e-a909628e5d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fedef4141151438283257c52844e4b7e, user_id=9a9c199b160444c2a4619c742a00ac31, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:25.855539 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f6312-5689-4c3e-8f6e-a909628e5d35 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:25.875652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1113/2255] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:25 2022] PUT /identity/v3/projects/fedef4141151438283257c52844e4b7e/users/9a9c199b160444c2a4619c742a00ac31/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:34:25.879791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f5b58-5567-41bf-af7d-5c6a0bb24397 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:25.897022 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f5b58-5567-41bf-af7d-5c6a0bb24397 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:26.024583 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b5534-3a2f-4fa9-8263-c8708e249054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:26.036083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b5534-3a2f-4fa9-8263-c8708e249054 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:26.036083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b5534-3a2f-4fa9-8263-c8708e249054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:26.036083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b5534-3a2f-4fa9-8263-c8708e249054 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:26.036083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b5534-3a2f-4fa9-8263-c8708e249054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:26.036083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b5534-3a2f-4fa9-8263-c8708e249054 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:26.071116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1143/2256] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:26.077249 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f6312-5689-4c3e-8f6e-a909628e5d35 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:26.091459 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f6312-5689-4c3e-8f6e-a909628e5d35 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:26.108250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7f38b-e555-47b4-8be9-d88aae85ece7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:26.108940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7f38b-e555-47b4-8be9-d88aae85ece7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:26.109223 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7f38b-e555-47b4-8be9-d88aae85ece7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:26.109531 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7f38b-e555-47b4-8be9-d88aae85ece7 admin admin] PATH_INFO: `/v3/projects/fedef4141151438283257c52844e4b7e/users/9a9c199b160444c2a4619c742a00ac31/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:26.120844 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7f38b-e555-47b4-8be9-d88aae85ece7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fedef4141151438283257c52844e4b7e, user_id=9a9c199b160444c2a4619c742a00ac31, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:26.124278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7f38b-e555-47b4-8be9-d88aae85ece7 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:26.156527 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1114/2257] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:26 2022] PUT /identity/v3/projects/fedef4141151438283257c52844e4b7e/users/9a9c199b160444c2a4619c742a00ac31/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:26.158101 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a0cb9-ea02-4901-94c9-f909bb8dec61 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:26.158394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a0cb9-ea02-4901-94c9-f909bb8dec61 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:26.158672 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a0cb9-ea02-4901-94c9-f909bb8dec61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:26.209459 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-401a0cb9-ea02-4901-94c9-f909bb8dec61 None None] MFA Rules not processed for user `9a9c199b160444c2a4619c742a00ac31`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:26.212287 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-401a0cb9-ea02-4901-94c9-f909bb8dec61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:26.321882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1144/2258] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:26.335585 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7f38b-e555-47b4-8be9-d88aae85ece7 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:26.337670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7af7f38b-e555-47b4-8be9-d88aae85ece7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:26.503560 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7f38b-e555-47b4-8be9-d88aae85ece7 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:26.507949 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9e1a0-e42e-4b42-8472-6c9f32a01dcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:26.508442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9e1a0-e42e-4b42-8472-6c9f32a01dcc service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:26.508827 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9e1a0-e42e-4b42-8472-6c9f32a01dcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:26.509199 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9e1a0-e42e-4b42-8472-6c9f32a01dcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:26.519813 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c9e1a0-e42e-4b42-8472-6c9f32a01dcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:26.521478 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c9e1a0-e42e-4b42-8472-6c9f32a01dcc service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:26.547102 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1115/2259] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:27.962670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a0cb9-ea02-4901-94c9-f909bb8dec61 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:28.094012 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:28.101606 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a0cb9-ea02-4901-94c9-f909bb8dec61 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:28.121635 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2fbbc-ca1c-4530-bde7-1ec521003eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:28.122299 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2fbbc-ca1c-4530-bde7-1ec521003eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:28.122580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2fbbc-ca1c-4530-bde7-1ec521003eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:28.122861 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2fbbc-ca1c-4530-bde7-1ec521003eea admin admin] PATH_INFO: `/v3/users/4f8b1706b3784751a230ea647de955b8` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:28.124783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2fbbc-ca1c-4530-bde7-1ec521003eea admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f8b1706b3784751a230ea647de955b8)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:28.126766 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b2fbbc-ca1c-4530-bde7-1ec521003eea admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:28.230256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-66b2fbbc-ca1c-4530-bde7-1ec521003eea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f8b1706b3784751a230ea647de955b8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:28.231880 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-66b2fbbc-ca1c-4530-bde7-1ec521003eea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f8b1706b3784751a230ea647de955b8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:28.248208 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-66b2fbbc-ca1c-4530-bde7-1ec521003eea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f8b1706b3784751a230ea647de955b8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:28.280588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-66b2fbbc-ca1c-4530-bde7-1ec521003eea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f8b1706b3784751a230ea647de955b8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:28.303515 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-66b2fbbc-ca1c-4530-bde7-1ec521003eea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f8b1706b3784751a230ea647de955b8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:28.321711 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1145/2260] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:27 2022] DELETE /identity/v3/users/4f8b1706b3784751a230ea647de955b8 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:28.326611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732b6af-01b8-4b46-ad5c-a687457f37a2 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:28.326611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732b6af-01b8-4b46-ad5c-a687457f37a2 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:28.327652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f732b6af-01b8-4b46-ad5c-a687457f37a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:28.365169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-f732b6af-01b8-4b46-ad5c-a687457f37a2 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:28.369984 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f732b6af-01b8-4b46-ad5c-a687457f37a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:28.394047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:28.531225 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1116/2261] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:28.535742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2fbbc-ca1c-4530-bde7-1ec521003eea admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:28.537183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-66b2fbbc-ca1c-4530-bde7-1ec521003eea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:28.674231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2fbbc-ca1c-4530-bde7-1ec521003eea admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:28.680712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca5e11-660b-44cf-ab86-1ada3043e834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:28.687757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca5e11-660b-44cf-ab86-1ada3043e834 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:28.687966 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca5e11-660b-44cf-ab86-1ada3043e834 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:28.688198 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca5e11-660b-44cf-ab86-1ada3043e834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:28.738463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca5e11-660b-44cf-ab86-1ada3043e834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:28.738463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca5e11-660b-44cf-ab86-1ada3043e834 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:28.749930 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1146/2262] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:29.342861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52a96d-077c-448d-81cc-592bc28076b3 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:29.343411 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52a96d-077c-448d-81cc-592bc28076b3 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:29.343411 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52a96d-077c-448d-81cc-592bc28076b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:29.386088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-5f52a96d-077c-448d-81cc-592bc28076b3 None None] MFA Rules not processed for user `e3df898326404ea4b1343fbcb5d70a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:29.387879 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-5f52a96d-077c-448d-81cc-592bc28076b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:29.507852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1117/2263] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:29.595957 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca5e11-660b-44cf-ab86-1ada3043e834 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:29.595957 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-41ca5e11-660b-44cf-ab86-1ada3043e834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:29.611815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca5e11-660b-44cf-ab86-1ada3043e834 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:29.614774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52a96d-077c-448d-81cc-592bc28076b3 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:29.617036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-5f52a96d-077c-448d-81cc-592bc28076b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:29.620067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10974580-4760-4f9c-9271-488fdeb26ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:29.620910 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10974580-4760-4f9c-9271-488fdeb26ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:29.621343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10974580-4760-4f9c-9271-488fdeb26ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:29.621928 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10974580-4760-4f9c-9271-488fdeb26ad0 admin admin] PATH_INFO: `/v3/projects/a4b96e33b1f64002bc1b75683bdab2e1` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:29.623982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10974580-4760-4f9c-9271-488fdeb26ad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4b96e33b1f64002bc1b75683bdab2e1)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:29.626612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10974580-4760-4f9c-9271-488fdeb26ad0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:29.714337 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-10974580-4760-4f9c-9271-488fdeb26ad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4b96e33b1f64002bc1b75683bdab2e1', 'request_id': 'req-10974580-4760-4f9c-9271-488fdeb26ad0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:29.740508 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-10974580-4760-4f9c-9271-488fdeb26ad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4b96e33b1f64002bc1b75683bdab2e1', 'request_id': 'req-10974580-4760-4f9c-9271-488fdeb26ad0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:29.756956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-10974580-4760-4f9c-9271-488fdeb26ad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4b96e33b1f64002bc1b75683bdab2e1', 'request_id': 'req-10974580-4760-4f9c-9271-488fdeb26ad0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:29.764933 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52a96d-077c-448d-81cc-592bc28076b3 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:29.786241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-10974580-4760-4f9c-9271-488fdeb26ad0 admin admin] The token cache is being invalidate because project a4b96e33b1f64002bc1b75683bdab2e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:29.786517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-10974580-4760-4f9c-9271-488fdeb26ad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:29.787437 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-10974580-4760-4f9c-9271-488fdeb26ad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:29.790049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1147/2264] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:29 2022] DELETE /identity/v3/projects/a4b96e33b1f64002bc1b75683bdab2e1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:29.793988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10974580-4760-4f9c-9271-488fdeb26ad0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:29.798010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-10974580-4760-4f9c-9271-488fdeb26ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:29.810267 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:29.841428 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:29.883001 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab0797-3218-4a82-bf8d-fd38d4ba8220 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:29.883785 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab0797-3218-4a82-bf8d-fd38d4ba8220 service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:29.884149 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab0797-3218-4a82-bf8d-fd38d4ba8220 service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:29.884495 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab0797-3218-4a82-bf8d-fd38d4ba8220 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:29.886578 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-faab0797-3218-4a82-bf8d-fd38d4ba8220 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:29.918825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10974580-4760-4f9c-9271-488fdeb26ad0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:29.924478 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6b390-3df8-442b-9226-0f906622521c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:29.925126 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6b390-3df8-442b-9226-0f906622521c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:29.925406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6b390-3df8-442b-9226-0f906622521c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:29.925688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6b390-3df8-442b-9226-0f906622521c admin admin] PATH_INFO: `/v3/users/9a9c199b160444c2a4619c742a00ac31` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:29.926946 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6b390-3df8-442b-9226-0f906622521c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a9c199b160444c2a4619c742a00ac31)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:29.928903 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6b390-3df8-442b-9226-0f906622521c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:29.974228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab0797-3218-4a82-bf8d-fd38d4ba8220 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:29.975992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab0797-3218-4a82-bf8d-fd38d4ba8220 service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:29.991548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1118/2265] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:30.017477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-dcb6b390-3df8-442b-9226-0f906622521c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a9c199b160444c2a4619c742a00ac31'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:30.018162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-dcb6b390-3df8-442b-9226-0f906622521c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a9c199b160444c2a4619c742a00ac31'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:30.026770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-dcb6b390-3df8-442b-9226-0f906622521c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a9c199b160444c2a4619c742a00ac31'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:30.040343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-dcb6b390-3df8-442b-9226-0f906622521c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a9c199b160444c2a4619c742a00ac31'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:30.065316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-dcb6b390-3df8-442b-9226-0f906622521c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a9c199b160444c2a4619c742a00ac31'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:30.075096 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1148/2266] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:29 2022] DELETE /identity/v3/users/9a9c199b160444c2a4619c742a00ac31 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:30.590617 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab0797-3218-4a82-bf8d-fd38d4ba8220 service nova] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:30.610679 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab0797-3218-4a82-bf8d-fd38d4ba8220 service nova] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:30.714294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2d01e-52e0-4204-af68-4ebf7a7921f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:30.714938 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2d01e-52e0-4204-af68-4ebf7a7921f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:30.715373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2d01e-52e0-4204-af68-4ebf7a7921f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:30.715678 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2d01e-52e0-4204-af68-4ebf7a7921f6 admin admin] PATH_INFO: `/v3/projects/fedef4141151438283257c52844e4b7e` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:30.716849 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2d01e-52e0-4204-af68-4ebf7a7921f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fedef4141151438283257c52844e4b7e)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:30.718922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2d01e-52e0-4204-af68-4ebf7a7921f6 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:30.785579 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2ae2d01e-52e0-4204-af68-4ebf7a7921f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fedef4141151438283257c52844e4b7e', 'request_id': 'req-2ae2d01e-52e0-4204-af68-4ebf7a7921f6'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:30.792575 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2ae2d01e-52e0-4204-af68-4ebf7a7921f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fedef4141151438283257c52844e4b7e', 'request_id': 'req-2ae2d01e-52e0-4204-af68-4ebf7a7921f6'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:30.802100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2ae2d01e-52e0-4204-af68-4ebf7a7921f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fedef4141151438283257c52844e4b7e', 'request_id': 'req-2ae2d01e-52e0-4204-af68-4ebf7a7921f6'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:30.815347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2ae2d01e-52e0-4204-af68-4ebf7a7921f6 admin admin] The token cache is being invalidate because project fedef4141151438283257c52844e4b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:30.815680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2ae2d01e-52e0-4204-af68-4ebf7a7921f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:30.816333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2ae2d01e-52e0-4204-af68-4ebf7a7921f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:30.819474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1119/2267] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:30 2022] DELETE /identity/v3/projects/fedef4141151438283257c52844e4b7e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:30.824130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d23a8-0685-46dc-a334-204e1e290b2d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:30.824616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d23a8-0685-46dc-a334-204e1e290b2d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:30.824716 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d23a8-0685-46dc-a334-204e1e290b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:30.848229 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-006d23a8-0685-46dc-a334-204e1e290b2d None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:30.850521 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-006d23a8-0685-46dc-a334-204e1e290b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:30.950377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1149/2268] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:30.952301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20ba13-9c1e-4967-ace8-4ffa3bf66b25 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:30.952588 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20ba13-9c1e-4967-ace8-4ffa3bf66b25 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:30.952898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20ba13-9c1e-4967-ace8-4ffa3bf66b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:30.976780 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:30.996480 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-7f20ba13-9c1e-4967-ace8-4ffa3bf66b25 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:30.996480 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7f20ba13-9c1e-4967-ace8-4ffa3bf66b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:31.012312 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1120/2269] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:31.017560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d23a8-0685-46dc-a334-204e1e290b2d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:31.027485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d23a8-0685-46dc-a334-204e1e290b2d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:31.035778 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e0e01-3e1b-4d4c-8f78-09859c211370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:31.036835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e0e01-3e1b-4d4c-8f78-09859c211370 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:31.036835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e0e01-3e1b-4d4c-8f78-09859c211370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:31.037180 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e0e01-3e1b-4d4c-8f78-09859c211370 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:31.037915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e0e01-3e1b-4d4c-8f78-09859c211370 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:31.038282 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e0e01-3e1b-4d4c-8f78-09859c211370 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:31.040015 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848e0e01-3e1b-4d4c-8f78-09859c211370 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:31.063542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1150/2270] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:34:31.067940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20ba13-9c1e-4967-ace8-4ffa3bf66b25 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:31.079410 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20ba13-9c1e-4967-ace8-4ffa3bf66b25 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:31.086045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2a08f-8c48-499d-9654-71de2ebd2b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:31.086234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2a08f-8c48-499d-9654-71de2ebd2b49 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:31.086574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2a08f-8c48-499d-9654-71de2ebd2b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:31.086891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2a08f-8c48-499d-9654-71de2ebd2b49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:31.088596 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2a08f-8c48-499d-9654-71de2ebd2b49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:31.090898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2a08f-8c48-499d-9654-71de2ebd2b49 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:31.128833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1121/2271] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:31 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:31.134078 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e0e01-3e1b-4d4c-8f78-09859c211370 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:31.149653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e0e01-3e1b-4d4c-8f78-09859c211370 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:31.156500 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:31.229964 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9add83f-4ee2-43fd-a578-0661881118dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:31.230820 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9add83f-4ee2-43fd-a578-0661881118dd admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:31.231157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9add83f-4ee2-43fd-a578-0661881118dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:31.231497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9add83f-4ee2-43fd-a578-0661881118dd admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:31.232688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9add83f-4ee2-43fd-a578-0661881118dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:31.234399 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9add83f-4ee2-43fd-a578-0661881118dd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:31.259444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1151/2272] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:31 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:31.261737 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2a08f-8c48-499d-9654-71de2ebd2b49 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:31.273443 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2a08f-8c48-499d-9654-71de2ebd2b49 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:31.279041 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd805ef-3f69-4c4f-ab6f-0cb63ef84378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:31.280324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd805ef-3f69-4c4f-ab6f-0cb63ef84378 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:31.280682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd805ef-3f69-4c4f-ab6f-0cb63ef84378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:31.281039 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd805ef-3f69-4c4f-ab6f-0cb63ef84378 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:31.281783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd805ef-3f69-4c4f-ab6f-0cb63ef84378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:31.283761 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd805ef-3f69-4c4f-ab6f-0cb63ef84378 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:31.295463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1122/2273] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:31.299348 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9add83f-4ee2-43fd-a578-0661881118dd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:31.310285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9add83f-4ee2-43fd-a578-0661881118dd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:31.327916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e01e5-eea2-4322-9660-168c15e84b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:31.328914 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e01e5-eea2-4322-9660-168c15e84b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:31.329318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e01e5-eea2-4322-9660-168c15e84b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:31.329668 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e01e5-eea2-4322-9660-168c15e84b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:31.331877 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e01e5-eea2-4322-9660-168c15e84b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:31.333849 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e01e5-eea2-4322-9660-168c15e84b77 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:31.350352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1152/2274] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:31.351141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd805ef-3f69-4c4f-ab6f-0cb63ef84378 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:31.365739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd805ef-3f69-4c4f-ab6f-0cb63ef84378 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:31.370904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe0bff-57cc-4b87-b28b-5d5f4a377f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:31.371753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe0bff-57cc-4b87-b28b-5d5f4a377f3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:31.372092 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe0bff-57cc-4b87-b28b-5d5f4a377f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:31.372446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe0bff-57cc-4b87-b28b-5d5f4a377f3b admin admin] PATH_INFO: `/v3/projects/5a6b7f68c0ac4d61a4cbaeadfe11fc28/users/998956bc32314b739d7657511d691c80/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:31.408997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe0bff-57cc-4b87-b28b-5d5f4a377f3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a6b7f68c0ac4d61a4cbaeadfe11fc28, user_id=998956bc32314b739d7657511d691c80, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:31.410233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe0bff-57cc-4b87-b28b-5d5f4a377f3b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:31.430648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1123/2275] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:31 2022] PUT /identity/v3/projects/5a6b7f68c0ac4d61a4cbaeadfe11fc28/users/998956bc32314b739d7657511d691c80/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:31.444921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c893fe-7857-4e82-ace6-070370a60287 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:31.444921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c893fe-7857-4e82-ace6-070370a60287 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:31.444921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c893fe-7857-4e82-ace6-070370a60287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:31.507480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-38c893fe-7857-4e82-ace6-070370a60287 None None] MFA Rules not processed for user `998956bc32314b739d7657511d691c80`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:31.509116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-38c893fe-7857-4e82-ace6-070370a60287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:31.591785 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1153/2276] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:31.600270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe0bff-57cc-4b87-b28b-5d5f4a377f3b admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:31.602303 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-8bfe0bff-57cc-4b87-b28b-5d5f4a377f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:31.704331 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe0bff-57cc-4b87-b28b-5d5f4a377f3b admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:31.708969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbf8bf-369c-4353-8aba-9e8436bf508f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:31.709741 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbf8bf-369c-4353-8aba-9e8436bf508f service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:31.710023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbf8bf-369c-4353-8aba-9e8436bf508f service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:31.710306 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbf8bf-369c-4353-8aba-9e8436bf508f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:31.724127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbf8bf-369c-4353-8aba-9e8436bf508f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:31.725582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbf8bf-369c-4353-8aba-9e8436bf508f service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:31.736519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1124/2277] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:40.447053 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c893fe-7857-4e82-ace6-070370a60287 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:40.527577 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c893fe-7857-4e82-ace6-070370a60287 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:40.533464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a115a4-d274-49e7-8d65-da3582b42f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:40.534752 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a115a4-d274-49e7-8d65-da3582b42f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:40.535038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a115a4-d274-49e7-8d65-da3582b42f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:40.535313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a115a4-d274-49e7-8d65-da3582b42f59 admin admin] PATH_INFO: `/v3/users/998956bc32314b739d7657511d691c80` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:40.536909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a115a4-d274-49e7-8d65-da3582b42f59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=998956bc32314b739d7657511d691c80)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:40.539454 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a115a4-d274-49e7-8d65-da3582b42f59 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:40.562694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:40.594492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-20a115a4-d274-49e7-8d65-da3582b42f59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '998956bc32314b739d7657511d691c80'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:40.595139 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-20a115a4-d274-49e7-8d65-da3582b42f59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '998956bc32314b739d7657511d691c80'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:40.602317 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-20a115a4-d274-49e7-8d65-da3582b42f59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '998956bc32314b739d7657511d691c80'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:40.610473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-20a115a4-d274-49e7-8d65-da3582b42f59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '998956bc32314b739d7657511d691c80'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:40.624393 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-20a115a4-d274-49e7-8d65-da3582b42f59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '998956bc32314b739d7657511d691c80'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:40.630666 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1154/2278] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:40 2022] DELETE /identity/v3/users/998956bc32314b739d7657511d691c80 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:40.635331 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdc8b8-5bb2-49c1-9f80-782122765e66 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:40.635928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdc8b8-5bb2-49c1-9f80-782122765e66 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:40.636133 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdc8b8-5bb2-49c1-9f80-782122765e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:40.655783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-33fdc8b8-5bb2-49c1-9f80-782122765e66 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:40.657001 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-33fdc8b8-5bb2-49c1-9f80-782122765e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:40.728356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1125/2279] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:40.745728 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a115a4-d274-49e7-8d65-da3582b42f59 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:40.747103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-20a115a4-d274-49e7-8d65-da3582b42f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:40.828930 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a115a4-d274-49e7-8d65-da3582b42f59 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:40.832847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4450b82-e839-4146-8d61-e73626e4a6c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:40.833450 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4450b82-e839-4146-8d61-e73626e4a6c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:40.833755 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4450b82-e839-4146-8d61-e73626e4a6c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:40.834028 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4450b82-e839-4146-8d61-e73626e4a6c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:40.846025 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4450b82-e839-4146-8d61-e73626e4a6c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:40.849858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4450b82-e839-4146-8d61-e73626e4a6c5 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:40.860555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1155/2280] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:40.867271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991eb757-7027-4025-9e2d-af72d0f41900 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:40.867557 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991eb757-7027-4025-9e2d-af72d0f41900 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:40.867886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991eb757-7027-4025-9e2d-af72d0f41900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:40.888861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-991eb757-7027-4025-9e2d-af72d0f41900 None None] MFA Rules not processed for user `9aa753d1e2c4430498d5b28f0304952f`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:40.889749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-991eb757-7027-4025-9e2d-af72d0f41900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:40.939861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:40.954951 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1126/2281] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:41.148400 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4450b82-e839-4146-8d61-e73626e4a6c5 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:41.150018 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f4450b82-e839-4146-8d61-e73626e4a6c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:41.165890 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4450b82-e839-4146-8d61-e73626e4a6c5 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:41.173375 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a687b-3f08-4666-9ba7-acb4f3e80d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:41.174470 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a687b-3f08-4666-9ba7-acb4f3e80d3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:41.174757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a687b-3f08-4666-9ba7-acb4f3e80d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:41.175131 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a687b-3f08-4666-9ba7-acb4f3e80d3b admin admin] PATH_INFO: `/v3/projects/5a6b7f68c0ac4d61a4cbaeadfe11fc28` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:41.176300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a687b-3f08-4666-9ba7-acb4f3e80d3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a6b7f68c0ac4d61a4cbaeadfe11fc28)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:41.179705 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a687b-3f08-4666-9ba7-acb4f3e80d3b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:41.240100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0c4a687b-3f08-4666-9ba7-acb4f3e80d3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a6b7f68c0ac4d61a4cbaeadfe11fc28', 'request_id': 'req-0c4a687b-3f08-4666-9ba7-acb4f3e80d3b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:41.246775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0c4a687b-3f08-4666-9ba7-acb4f3e80d3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a6b7f68c0ac4d61a4cbaeadfe11fc28', 'request_id': 'req-0c4a687b-3f08-4666-9ba7-acb4f3e80d3b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:41.258483 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0c4a687b-3f08-4666-9ba7-acb4f3e80d3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a6b7f68c0ac4d61a4cbaeadfe11fc28', 'request_id': 'req-0c4a687b-3f08-4666-9ba7-acb4f3e80d3b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:41.270075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0c4a687b-3f08-4666-9ba7-acb4f3e80d3b admin admin] The token cache is being invalidate because project 5a6b7f68c0ac4d61a4cbaeadfe11fc28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:41.270394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0c4a687b-3f08-4666-9ba7-acb4f3e80d3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:41.271215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-0c4a687b-3f08-4666-9ba7-acb4f3e80d3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:41.273590 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1156/2282] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:41 2022] DELETE /identity/v3/projects/5a6b7f68c0ac4d61a4cbaeadfe11fc28 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:41.277963 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05655167-7861-417d-9d84-0d67f136002b None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:41.278419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05655167-7861-417d-9d84-0d67f136002b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:41.278759 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05655167-7861-417d-9d84-0d67f136002b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:41.299985 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-05655167-7861-417d-9d84-0d67f136002b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:41.301294 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-05655167-7861-417d-9d84-0d67f136002b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:41.364528 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1127/2283] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:41.372371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eca79-ffe1-42d9-90ec-9286b59e5e39 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:41.372814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eca79-ffe1-42d9-90ec-9286b59e5e39 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:41.373190 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eca79-ffe1-42d9-90ec-9286b59e5e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:41.394299 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a86eca79-ffe1-42d9-90ec-9286b59e5e39 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:41.395567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a86eca79-ffe1-42d9-90ec-9286b59e5e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:41.404636 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1157/2284] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:41.409034 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05655167-7861-417d-9d84-0d67f136002b None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:41.418702 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05655167-7861-417d-9d84-0d67f136002b None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:41.424307 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdcc00-dac0-4278-8b7a-95357e15b63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:41.424923 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdcc00-dac0-4278-8b7a-95357e15b63b admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:41.425202 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdcc00-dac0-4278-8b7a-95357e15b63b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:41.425471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdcc00-dac0-4278-8b7a-95357e15b63b admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:41.426225 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdcc00-dac0-4278-8b7a-95357e15b63b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:41.426542 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdcc00-dac0-4278-8b7a-95357e15b63b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:41.428277 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdcc00-dac0-4278-8b7a-95357e15b63b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:41.441410 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1128/2285] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:34:41.446282 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86eca79-ffe1-42d9-90ec-9286b59e5e39 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:41.457163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86eca79-ffe1-42d9-90ec-9286b59e5e39 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:41.462164 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798232c-c514-405f-93c2-33dc4274dbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:41.462810 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798232c-c514-405f-93c2-33dc4274dbd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:41.463091 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798232c-c514-405f-93c2-33dc4274dbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:41.463414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798232c-c514-405f-93c2-33dc4274dbd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:41.464085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798232c-c514-405f-93c2-33dc4274dbd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:41.466009 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798232c-c514-405f-93c2-33dc4274dbd2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:41.483006 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:41.496989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1158/2286] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:41 2022] POST /identity/v3/projects => generated 359 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:34:41.501361 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdcc00-dac0-4278-8b7a-95357e15b63b admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:41.521441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdcc00-dac0-4278-8b7a-95357e15b63b admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:41.581264 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b1039-ef2c-46c5-9180-01488f8286e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:41.581881 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b1039-ef2c-46c5-9180-01488f8286e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:41.582161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b1039-ef2c-46c5-9180-01488f8286e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:41.582546 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b1039-ef2c-46c5-9180-01488f8286e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:41.583188 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b1039-ef2c-46c5-9180-01488f8286e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:41.585146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b1039-ef2c-46c5-9180-01488f8286e5 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:41.606906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1129/2287] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:41 2022] POST /identity/v3/users => generated 280 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:41.611167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798232c-c514-405f-93c2-33dc4274dbd2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:41.621409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798232c-c514-405f-93c2-33dc4274dbd2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:41.626447 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7676a4a-53b3-479b-9ea7-356e8e423812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:41.627086 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7676a4a-53b3-479b-9ea7-356e8e423812 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:41.627368 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7676a4a-53b3-479b-9ea7-356e8e423812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:41.628137 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7676a4a-53b3-479b-9ea7-356e8e423812 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:41.628308 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7676a4a-53b3-479b-9ea7-356e8e423812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:41.629906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7676a4a-53b3-479b-9ea7-356e8e423812 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:41.642637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1159/2288] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:41.645639 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b1039-ef2c-46c5-9180-01488f8286e5 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:41.656824 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b1039-ef2c-46c5-9180-01488f8286e5 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:41.662018 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3b43e-04b3-44c1-9a0f-2b809403f59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:41.662651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3b43e-04b3-44c1-9a0f-2b809403f59e admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:41.662936 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3b43e-04b3-44c1-9a0f-2b809403f59e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:41.663238 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3b43e-04b3-44c1-9a0f-2b809403f59e admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:41.663976 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3b43e-04b3-44c1-9a0f-2b809403f59e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:41.665576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3b43e-04b3-44c1-9a0f-2b809403f59e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:41.678569 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1130/2289] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:41.682995 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7676a4a-53b3-479b-9ea7-356e8e423812 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:41.693670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7676a4a-53b3-479b-9ea7-356e8e423812 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:41.698428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2e917-0b8e-41f0-a858-5a666eb3617b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:41.699159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2e917-0b8e-41f0-a858-5a666eb3617b admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:41.699422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2e917-0b8e-41f0-a858-5a666eb3617b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:41.699726 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2e917-0b8e-41f0-a858-5a666eb3617b admin admin] PATH_INFO: `/v3/projects/f965488acf104ebb96ecc2ffd62b2eb0/users/f451180710f84537b09f930babdfa3a8/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:41.728882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2e917-0b8e-41f0-a858-5a666eb3617b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f965488acf104ebb96ecc2ffd62b2eb0, user_id=f451180710f84537b09f930babdfa3a8, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:41.731927 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb2e917-0b8e-41f0-a858-5a666eb3617b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:41.745492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1160/2290] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:41 2022] PUT /identity/v3/projects/f965488acf104ebb96ecc2ffd62b2eb0/users/f451180710f84537b09f930babdfa3a8/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:41.756806 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0708cc-57a2-4357-a25d-6a68474810d1 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:41.757097 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0708cc-57a2-4357-a25d-6a68474810d1 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:41.757374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0708cc-57a2-4357-a25d-6a68474810d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:41.785676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-fc0708cc-57a2-4357-a25d-6a68474810d1 None None] MFA Rules not processed for user `f451180710f84537b09f930babdfa3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:41.786892 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fc0708cc-57a2-4357-a25d-6a68474810d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:41.839645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:41.860860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1131/2291] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:41.903110 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2e917-0b8e-41f0-a858-5a666eb3617b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:41.905649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cfb2e917-0b8e-41f0-a858-5a666eb3617b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:41.965127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2e917-0b8e-41f0-a858-5a666eb3617b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:41.968992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e81d98-917c-44c3-ac66-3ca93942fe24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:41.969607 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e81d98-917c-44c3-ac66-3ca93942fe24 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:41.969888 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e81d98-917c-44c3-ac66-3ca93942fe24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:41.970161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e81d98-917c-44c3-ac66-3ca93942fe24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:41.978651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e81d98-917c-44c3-ac66-3ca93942fe24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:41.980121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e81d98-917c-44c3-ac66-3ca93942fe24 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:41.988852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1161/2292] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:41 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:42.005698 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0708cc-57a2-4357-a25d-6a68474810d1 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:42.069728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0708cc-57a2-4357-a25d-6a68474810d1 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:42.074825 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6d148-22c9-4b6b-945a-5d5f68fcce94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:42.075641 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6d148-22c9-4b6b-945a-5d5f68fcce94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:42.075983 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6d148-22c9-4b6b-945a-5d5f68fcce94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:42.076265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6d148-22c9-4b6b-945a-5d5f68fcce94 admin admin] PATH_INFO: `/v3/users/f451180710f84537b09f930babdfa3a8` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:42.078532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6d148-22c9-4b6b-945a-5d5f68fcce94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f451180710f84537b09f930babdfa3a8)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:42.080936 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6d148-22c9-4b6b-945a-5d5f68fcce94 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:42.132540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b3f6d148-22c9-4b6b-945a-5d5f68fcce94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f451180710f84537b09f930babdfa3a8'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:42.133127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b3f6d148-22c9-4b6b-945a-5d5f68fcce94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f451180710f84537b09f930babdfa3a8'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:42.133731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b3f6d148-22c9-4b6b-945a-5d5f68fcce94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f451180710f84537b09f930babdfa3a8'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:42.141751 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b3f6d148-22c9-4b6b-945a-5d5f68fcce94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f451180710f84537b09f930babdfa3a8'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:42.150662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-b3f6d148-22c9-4b6b-945a-5d5f68fcce94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f451180710f84537b09f930babdfa3a8'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:42.174721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1132/2293] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:42 2022] DELETE /identity/v3/users/f451180710f84537b09f930babdfa3a8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:42.180306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d4642-f30a-41b2-956c-99687b667709 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:42.180614 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d4642-f30a-41b2-956c-99687b667709 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:42.180890 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d4642-f30a-41b2-956c-99687b667709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:42.204682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-432d4642-f30a-41b2-956c-99687b667709 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:42.205886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-432d4642-f30a-41b2-956c-99687b667709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:42.269053 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1162/2294] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:42.279878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6d148-22c9-4b6b-945a-5d5f68fcce94 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:42.285561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b3f6d148-22c9-4b6b-945a-5d5f68fcce94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:42.352718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6d148-22c9-4b6b-945a-5d5f68fcce94 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:42.356481 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be8791-aed0-4da9-96d2-8893d8913264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:42.357595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be8791-aed0-4da9-96d2-8893d8913264 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:42.357876 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be8791-aed0-4da9-96d2-8893d8913264 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:42.358152 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be8791-aed0-4da9-96d2-8893d8913264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:42.375503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be8791-aed0-4da9-96d2-8893d8913264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:42.377766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63be8791-aed0-4da9-96d2-8893d8913264 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:42.388434 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1133/2295] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:42.724328 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d4642-f30a-41b2-956c-99687b667709 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:42.725890 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-432d4642-f30a-41b2-956c-99687b667709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:42.743454 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d4642-f30a-41b2-956c-99687b667709 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:42.763026 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea145c14-a77c-437b-a4b7-e01c863ed646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:42.763849 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea145c14-a77c-437b-a4b7-e01c863ed646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:42.764124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea145c14-a77c-437b-a4b7-e01c863ed646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:42.765320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea145c14-a77c-437b-a4b7-e01c863ed646 admin admin] PATH_INFO: `/v3/projects/f965488acf104ebb96ecc2ffd62b2eb0` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:42.766515 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea145c14-a77c-437b-a4b7-e01c863ed646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f965488acf104ebb96ecc2ffd62b2eb0)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:42.768551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea145c14-a77c-437b-a4b7-e01c863ed646 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:42.829137 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ea145c14-a77c-437b-a4b7-e01c863ed646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f965488acf104ebb96ecc2ffd62b2eb0', 'request_id': 'req-ea145c14-a77c-437b-a4b7-e01c863ed646'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:42.834195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:INSERT=2,keystone:SELECT=92,keystone:DELETE=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:42.835744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ea145c14-a77c-437b-a4b7-e01c863ed646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f965488acf104ebb96ecc2ffd62b2eb0', 'request_id': 'req-ea145c14-a77c-437b-a4b7-e01c863ed646'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:42.845510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ea145c14-a77c-437b-a4b7-e01c863ed646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f965488acf104ebb96ecc2ffd62b2eb0', 'request_id': 'req-ea145c14-a77c-437b-a4b7-e01c863ed646'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:42.859731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ea145c14-a77c-437b-a4b7-e01c863ed646 admin admin] The token cache is being invalidate because project f965488acf104ebb96ecc2ffd62b2eb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:42.859731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ea145c14-a77c-437b-a4b7-e01c863ed646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:42.860313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-ea145c14-a77c-437b-a4b7-e01c863ed646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:42.862033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1163/2296] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:42 2022] DELETE /identity/v3/projects/f965488acf104ebb96ecc2ffd62b2eb0 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:46.131329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eec98a-9459-49dd-aca2-a17fdc78f7e0 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:46.131822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eec98a-9459-49dd-aca2-a17fdc78f7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:46.131913 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eec98a-9459-49dd-aca2-a17fdc78f7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:46.154199 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-18eec98a-9459-49dd-aca2-a17fdc78f7e0 None None] MFA Rules not processed for user `9138ac21bbdb4577a7f43b5ab471d572`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:46.156068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-18eec98a-9459-49dd-aca2-a17fdc78f7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:46.161424 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:46.267815 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1134/2297] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:52.854163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=5,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:53.458452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436241f-fb29-4c73-91ec-266583d31757 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:53.458741 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436241f-fb29-4c73-91ec-266583d31757 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:53.459049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436241f-fb29-4c73-91ec-266583d31757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:53.482689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2436241f-fb29-4c73-91ec-266583d31757 None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:53.484200 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2436241f-fb29-4c73-91ec-266583d31757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:53.564964 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1164/2298] 10.222.0.58 () {58 vars in 1009 bytes} [Wed Aug 24 12:34:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:53.572762 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eec98a-9459-49dd-aca2-a17fdc78f7e0 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:53.574715 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-18eec98a-9459-49dd-aca2-a17fdc78f7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:53.673405 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eec98a-9459-49dd-aca2-a17fdc78f7e0 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:53.673405 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d9290-6b21-4772-8aa9-da70cb79136c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:53.673405 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d9290-6b21-4772-8aa9-da70cb79136c service swift] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:53.673405 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d9290-6b21-4772-8aa9-da70cb79136c service swift] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:53.673858 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d9290-6b21-4772-8aa9-da70cb79136c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:53.677863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d9290-6b21-4772-8aa9-da70cb79136c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:53.679503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d9290-6b21-4772-8aa9-da70cb79136c service swift] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:53.688315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1135/2299] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:34:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:34:53.834673 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436241f-fb29-4c73-91ec-266583d31757 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:53.836242 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2436241f-fb29-4c73-91ec-266583d31757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:53.951396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436241f-fb29-4c73-91ec-266583d31757 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:53.969082 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fb12e-a21b-4460-8478-83a3277d9e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:53.969869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fb12e-a21b-4460-8478-83a3277d9e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:53.971748 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fb12e-a21b-4460-8478-83a3277d9e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:53.972039 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fb12e-a21b-4460-8478-83a3277d9e93 admin admin] PATH_INFO: `/v3/users/95680494f6e54d8f83467f52be785c1b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:53.974678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fb12e-a21b-4460-8478-83a3277d9e93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95680494f6e54d8f83467f52be785c1b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:53.976792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fb12e-a21b-4460-8478-83a3277d9e93 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:54.086941 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c48fb12e-a21b-4460-8478-83a3277d9e93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95680494f6e54d8f83467f52be785c1b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:54.088531 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c48fb12e-a21b-4460-8478-83a3277d9e93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95680494f6e54d8f83467f52be785c1b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:54.104299 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c48fb12e-a21b-4460-8478-83a3277d9e93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95680494f6e54d8f83467f52be785c1b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:54.121835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c48fb12e-a21b-4460-8478-83a3277d9e93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95680494f6e54d8f83467f52be785c1b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:54.137753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c48fb12e-a21b-4460-8478-83a3277d9e93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95680494f6e54d8f83467f52be785c1b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:54.149452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1165/2300] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:34:53 2022] DELETE /identity/v3/users/95680494f6e54d8f83467f52be785c1b => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:54.154800 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e18c2-e2ab-4207-a86f-4d5d57ab4873 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:54.155140 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e18c2-e2ab-4207-a86f-4d5d57ab4873 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:54.155471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e18c2-e2ab-4207-a86f-4d5d57ab4873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:54.175485 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-b85e18c2-e2ab-4207-a86f-4d5d57ab4873 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:54.176941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b85e18c2-e2ab-4207-a86f-4d5d57ab4873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:54.277771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1136/2301] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:54.285639 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fb12e-a21b-4460-8478-83a3277d9e93 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:54.287438 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c48fb12e-a21b-4460-8478-83a3277d9e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:54.357434 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:54.425152 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fb12e-a21b-4460-8478-83a3277d9e93 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:54.430358 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d68aff-c7f9-43ba-b992-64852fe22d35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:54.431062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d68aff-c7f9-43ba-b992-64852fe22d35 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:54.431351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d68aff-c7f9-43ba-b992-64852fe22d35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:54.431747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d68aff-c7f9-43ba-b992-64852fe22d35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:54.443825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d68aff-c7f9-43ba-b992-64852fe22d35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:54.445679 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d68aff-c7f9-43ba-b992-64852fe22d35 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:54.463814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1166/2302] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:34:54.737588 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591ace2-8f91-472f-a82d-c2595ae09970 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:54.738157 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591ace2-8f91-472f-a82d-c2595ae09970 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:54.738549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591ace2-8f91-472f-a82d-c2595ae09970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:54.821992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-2591ace2-8f91-472f-a82d-c2595ae09970 None None] MFA Rules not processed for user `18f8f419b523440f8a4ca06dc83bff40`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:54.823593 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2591ace2-8f91-472f-a82d-c2595ae09970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:54.906226 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:54.935748 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d68aff-c7f9-43ba-b992-64852fe22d35 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:54.937484 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-29d68aff-c7f9-43ba-b992-64852fe22d35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:54.974625 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d68aff-c7f9-43ba-b992-64852fe22d35 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:54.982807 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1137/2303] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:54.996188 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf9cc11-0106-4123-ae80-56567bb4affd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:54.997002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9cc11-0106-4123-ae80-56567bb4affd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:54.997393 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9cc11-0106-4123-ae80-56567bb4affd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:54.999092 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf9cc11-0106-4123-ae80-56567bb4affd admin admin] PATH_INFO: `/v3/projects/910593fbebc941919de6d26feae6cac6` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.011389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9cc11-0106-4123-ae80-56567bb4affd admin admin] RBAC: Authorizing `identity:delete_project(project_id=910593fbebc941919de6d26feae6cac6)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:55.015815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf9cc11-0106-4123-ae80-56567bb4affd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:55.108784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7cf9cc11-0106-4123-ae80-56567bb4affd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '910593fbebc941919de6d26feae6cac6', 'request_id': 'req-7cf9cc11-0106-4123-ae80-56567bb4affd'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:55.115943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7cf9cc11-0106-4123-ae80-56567bb4affd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '910593fbebc941919de6d26feae6cac6', 'request_id': 'req-7cf9cc11-0106-4123-ae80-56567bb4affd'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:55.124771 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7cf9cc11-0106-4123-ae80-56567bb4affd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '910593fbebc941919de6d26feae6cac6', 'request_id': 'req-7cf9cc11-0106-4123-ae80-56567bb4affd'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:55.147899 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7cf9cc11-0106-4123-ae80-56567bb4affd admin admin] The token cache is being invalidate because project 910593fbebc941919de6d26feae6cac6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:34:55.148241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7cf9cc11-0106-4123-ae80-56567bb4affd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:55.149917 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7cf9cc11-0106-4123-ae80-56567bb4affd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:34:55.156355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1167/2304] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:34:54 2022] DELETE /identity/v3/projects/910593fbebc941919de6d26feae6cac6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:55.157835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1efc6-c9a7-42c2-85f3-327d913bbbd1 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.158219 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1efc6-c9a7-42c2-85f3-327d913bbbd1 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.158382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1efc6-c9a7-42c2-85f3-327d913bbbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.191243 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-e7c1efc6-c9a7-42c2-85f3-327d913bbbd1 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:55.192511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e7c1efc6-c9a7-42c2-85f3-327d913bbbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:55.325175 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1138/2305] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:34:55.333672 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6299348-3c9b-4d4e-b3af-9e2193116064 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.333959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6299348-3c9b-4d4e-b3af-9e2193116064 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.334237 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6299348-3c9b-4d4e-b3af-9e2193116064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.373200 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d6299348-3c9b-4d4e-b3af-9e2193116064 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:55.389385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d6299348-3c9b-4d4e-b3af-9e2193116064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:55.408582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1efc6-c9a7-42c2-85f3-327d913bbbd1 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:55.418294 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1168/2306] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:55.424575 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1efc6-c9a7-42c2-85f3-327d913bbbd1 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:55.445887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f323b4-97b0-465f-850b-7c2b93d761c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:55.447605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f323b4-97b0-465f-850b-7c2b93d761c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.448179 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f323b4-97b0-465f-850b-7c2b93d761c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.449806 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f323b4-97b0-465f-850b-7c2b93d761c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.452269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f323b4-97b0-465f-850b-7c2b93d761c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:34:55.452767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f323b4-97b0-465f-850b-7c2b93d761c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:55.454649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f323b4-97b0-465f-850b-7c2b93d761c5 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:55.475731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6299348-3c9b-4d4e-b3af-9e2193116064 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:55.489711 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1139/2307] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:34:55.493701 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6299348-3c9b-4d4e-b3af-9e2193116064 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:55.502122 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb945fc-91a8-49d2-aaf8-003188f2d4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:55.503880 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb945fc-91a8-49d2-aaf8-003188f2d4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.504373 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb945fc-91a8-49d2-aaf8-003188f2d4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.504867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb945fc-91a8-49d2-aaf8-003188f2d4a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.506256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb945fc-91a8-49d2-aaf8-003188f2d4a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:55.508864 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb945fc-91a8-49d2-aaf8-003188f2d4a9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:55.607671 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1169/2308] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:34:55 2022] POST /identity/v3/projects => generated 389 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:34:55.609571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f323b4-97b0-465f-850b-7c2b93d761c5 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:55.624068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f323b4-97b0-465f-850b-7c2b93d761c5 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:55.712654 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6651b-7ee3-423d-a85f-a70ad8d44b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:55.715258 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6651b-7ee3-423d-a85f-a70ad8d44b31 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.715544 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6651b-7ee3-423d-a85f-a70ad8d44b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.715919 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6651b-7ee3-423d-a85f-a70ad8d44b31 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.716545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b6651b-7ee3-423d-a85f-a70ad8d44b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:55.719478 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b6651b-7ee3-423d-a85f-a70ad8d44b31 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:55.755934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1140/2309] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:34:55 2022] POST /identity/v3/users => generated 295 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:34:55.759041 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb945fc-91a8-49d2-aaf8-003188f2d4a9 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:55.778326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb945fc-91a8-49d2-aaf8-003188f2d4a9 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:55.792285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb4429-f161-4332-be0b-fcbcb6f314b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:55.793443 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb4429-f161-4332-be0b-fcbcb6f314b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.793960 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb4429-f161-4332-be0b-fcbcb6f314b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.795093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb4429-f161-4332-be0b-fcbcb6f314b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.795909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb4429-f161-4332-be0b-fcbcb6f314b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:55.798311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb4429-f161-4332-be0b-fcbcb6f314b1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:55.832287 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1170/2310] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:34:55.834406 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6651b-7ee3-423d-a85f-a70ad8d44b31 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:55.846746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6651b-7ee3-423d-a85f-a70ad8d44b31 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:55.861685 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbf785-a0a3-45f7-9cb1-d33cd47eea6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:55.864300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbf785-a0a3-45f7-9cb1-d33cd47eea6b admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.864536 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbf785-a0a3-45f7-9cb1-d33cd47eea6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.864893 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbf785-a0a3-45f7-9cb1-d33cd47eea6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.866806 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbf785-a0a3-45f7-9cb1-d33cd47eea6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:55.869251 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbf785-a0a3-45f7-9cb1-d33cd47eea6b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:55.885624 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1141/2311] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:34:55.890394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb4429-f161-4332-be0b-fcbcb6f314b1 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:55.905881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb4429-f161-4332-be0b-fcbcb6f314b1 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:55.912574 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af2e64-6ab8-47e5-b549-85d72f7baa6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:55.913242 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af2e64-6ab8-47e5-b549-85d72f7baa6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.913517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af2e64-6ab8-47e5-b549-85d72f7baa6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.913791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af2e64-6ab8-47e5-b549-85d72f7baa6c admin admin] PATH_INFO: `/v3/projects/8bdb939f77eb4cf0ac73c4aac8a42036/users/4fe6905c83604c47af350d66e64f45e0/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:55.955829 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af2e64-6ab8-47e5-b549-85d72f7baa6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bdb939f77eb4cf0ac73c4aac8a42036, user_id=4fe6905c83604c47af350d66e64f45e0, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:55.961836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af2e64-6ab8-47e5-b549-85d72f7baa6c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:55.993416 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1171/2312] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:34:55 2022] PUT /identity/v3/projects/8bdb939f77eb4cf0ac73c4aac8a42036/users/4fe6905c83604c47af350d66e64f45e0/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:34:55.998097 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dda66-bc19-41e6-a69e-cec2acb90c4e None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:55.998676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dda66-bc19-41e6-a69e-cec2acb90c4e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:55.999195 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dda66-bc19-41e6-a69e-cec2acb90c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:56.037108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-8e5dda66-bc19-41e6-a69e-cec2acb90c4e None None] MFA Rules not processed for user `4fe6905c83604c47af350d66e64f45e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:34:56.038573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-8e5dda66-bc19-41e6-a69e-cec2acb90c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:56.111982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:56.147464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1142/2313] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:34:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:34:56.161915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af2e64-6ab8-47e5-b549-85d72f7baa6c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:34:56.164500 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-75af2e64-6ab8-47e5-b549-85d72f7baa6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:34:56.310385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:34:56.326220 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af2e64-6ab8-47e5-b549-85d72f7baa6c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:34:56.339037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae92d017-6e5c-4a99-a49c-b65fbfd70860 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:34:56.339037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92d017-6e5c-4a99-a49c-b65fbfd70860 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:34:56.339037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92d017-6e5c-4a99-a49c-b65fbfd70860 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:34:56.339037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae92d017-6e5c-4a99-a49c-b65fbfd70860 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:34:56.379057 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92d017-6e5c-4a99-a49c-b65fbfd70860 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:34:56.381784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae92d017-6e5c-4a99-a49c-b65fbfd70860 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:34:56.416913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1172/2314] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:34:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:35:06.143887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=7 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:06.390716 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:14.330651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2b134-8140-4ee0-8a16-9a738d281265 None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:14.331273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2b134-8140-4ee0-8a16-9a738d281265 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:14.331273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2b134-8140-4ee0-8a16-9a738d281265 None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:14.332704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1143/2315] 10.222.0.58 () {54 vars in 921 bytes} [Wed Aug 24 12:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:35:14.339449 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a25cea-f7d2-48ee-b647-366b106db37c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:14.339984 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a25cea-f7d2-48ee-b647-366b106db37c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:14.340164 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a25cea-f7d2-48ee-b647-366b106db37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:14.362498 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f8a25cea-f7d2-48ee-b647-366b106db37c None None] MFA Rules not processed for user `311dbed3ab0b44a7830a93227ff9ad33`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:14.363972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f8a25cea-f7d2-48ee-b647-366b106db37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:14.443410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1173/2316] 10.222.0.58 () {58 vars in 1018 bytes} [Wed Aug 24 12:35:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:35:14.455789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2b134-8140-4ee0-8a16-9a738d281265 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:14.457063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-91f2b134-8140-4ee0-8a16-9a738d281265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:14.530903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2b134-8140-4ee0-8a16-9a738d281265 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:14.534674 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117507f-2e17-4a1e-ad2c-e89a15f75a7a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd8df2f3357d4660bc22366f9746f89c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:14.535279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117507f-2e17-4a1e-ad2c-e89a15f75a7a service glance] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:14.535559 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117507f-2e17-4a1e-ad2c-e89a15f75a7a service glance] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:14.535984 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117507f-2e17-4a1e-ad2c-e89a15f75a7a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:14.547138 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117507f-2e17-4a1e-ad2c-e89a15f75a7a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:14.548617 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117507f-2e17-4a1e-ad2c-e89a15f75a7a service glance] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:14.559652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1144/2317] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:35:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:35:14.638615 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c305b-2acf-44fa-b2da-2e0b4ffcc473 None None] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:14.638922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c305b-2acf-44fa-b2da-2e0b4ffcc473 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:14.639194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9c305b-2acf-44fa-b2da-2e0b4ffcc473 None None] PATH_INFO: `/` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:14.640724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1174/2318] 10.222.0.58 () {54 vars in 921 bytes} [Wed Aug 24 12:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:35:14.645329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85388b-5529-418e-b939-1955ebd45563 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:14.645699 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85388b-5529-418e-b939-1955ebd45563 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:14.646045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85388b-5529-418e-b939-1955ebd45563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:14.667262 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-bd85388b-5529-418e-b939-1955ebd45563 None None] MFA Rules not processed for user `34fcb5885325472bbe338cf05d3788f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:14.669102 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-bd85388b-5529-418e-b939-1955ebd45563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:14.722961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1145/2319] 10.222.0.58 () {58 vars in 1018 bytes} [Wed Aug 24 12:35:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:35:14.735414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c305b-2acf-44fa-b2da-2e0b4ffcc473 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:14.737161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3d9c305b-2acf-44fa-b2da-2e0b4ffcc473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:14.750605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9c305b-2acf-44fa-b2da-2e0b4ffcc473 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:14.754097 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5663ba-b9ab-4c8d-aadd-2dc7f71bc1c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:14.754695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5663ba-b9ab-4c8d-aadd-2dc7f71bc1c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:14.754973 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5663ba-b9ab-4c8d-aadd-2dc7f71bc1c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:14.755244 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5663ba-b9ab-4c8d-aadd-2dc7f71bc1c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:14.762332 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5663ba-b9ab-4c8d-aadd-2dc7f71bc1c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:14.763791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5663ba-b9ab-4c8d-aadd-2dc7f71bc1c6 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:14.771384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1175/2320] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:35:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:35:14.854890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f639a-c0f8-4537-a663-dfd6ebf8241b None None] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:14.855181 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f639a-c0f8-4537-a663-dfd6ebf8241b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:14.855459 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f639a-c0f8-4537-a663-dfd6ebf8241b None None] PATH_INFO: `/` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:14.857537 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1146/2321] 10.222.0.58 () {54 vars in 921 bytes} [Wed Aug 24 12:35:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 12:35:14.863092 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51a057-6ab2-4bbb-bb24-c4a50c28c52c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:14.863385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51a057-6ab2-4bbb-bb24-c4a50c28c52c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:14.863698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51a057-6ab2-4bbb-bb24-c4a50c28c52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:14.883657 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-fd51a057-6ab2-4bbb-bb24-c4a50c28c52c None None] MFA Rules not processed for user `311dbed3ab0b44a7830a93227ff9ad33`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:14.884865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fd51a057-6ab2-4bbb-bb24-c4a50c28c52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:14.894744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1176/2322] 10.222.0.58 () {58 vars in 1018 bytes} [Wed Aug 24 12:35:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:35:14.906680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f639a-c0f8-4537-a663-dfd6ebf8241b None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:14.917860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f639a-c0f8-4537-a663-dfd6ebf8241b None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:14.921753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b55f8b-d0ae-4982-b4bf-335ca434408f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:14.922355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b55f8b-d0ae-4982-b4bf-335ca434408f service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:14.922638 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b55f8b-d0ae-4982-b4bf-335ca434408f service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:14.922913 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b55f8b-d0ae-4982-b4bf-335ca434408f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:14.932906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b55f8b-d0ae-4982-b4bf-335ca434408f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:14.934552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b55f8b-d0ae-4982-b4bf-335ca434408f service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:14.945092 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1147/2323] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:35:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:35:16.413744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51a057-6ab2-4bbb-bb24-c4a50c28c52c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:16.478646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51a057-6ab2-4bbb-bb24-c4a50c28c52c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:16.483448 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740dbb9-4bf4-438e-a366-bd079d82fbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:16.484120 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740dbb9-4bf4-438e-a366-bd079d82fbd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:16.484408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740dbb9-4bf4-438e-a366-bd079d82fbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:16.484714 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740dbb9-4bf4-438e-a366-bd079d82fbd2 admin admin] PATH_INFO: `/v3/users/4fe6905c83604c47af350d66e64f45e0` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:16.485849 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9740dbb9-4bf4-438e-a366-bd079d82fbd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fe6905c83604c47af350d66e64f45e0)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:16.487836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9740dbb9-4bf4-438e-a366-bd079d82fbd2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:16.536944 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9740dbb9-4bf4-438e-a366-bd079d82fbd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fe6905c83604c47af350d66e64f45e0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:16.537603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9740dbb9-4bf4-438e-a366-bd079d82fbd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fe6905c83604c47af350d66e64f45e0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:16.544730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9740dbb9-4bf4-438e-a366-bd079d82fbd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fe6905c83604c47af350d66e64f45e0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:16.551973 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9740dbb9-4bf4-438e-a366-bd079d82fbd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fe6905c83604c47af350d66e64f45e0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:16.560187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:16.565249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9740dbb9-4bf4-438e-a366-bd079d82fbd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fe6905c83604c47af350d66e64f45e0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:16.572409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1177/2324] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:35:16 2022] DELETE /identity/v3/users/4fe6905c83604c47af350d66e64f45e0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:35:16.577811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947294c-b368-4526-8714-4de8b39ee9c4 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:16.578302 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947294c-b368-4526-8714-4de8b39ee9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:16.578395 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947294c-b368-4526-8714-4de8b39ee9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:16.598066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-7947294c-b368-4526-8714-4de8b39ee9c4 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:16.599268 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7947294c-b368-4526-8714-4de8b39ee9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:16.657763 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1148/2325] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:35:16.667735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740dbb9-4bf4-438e-a366-bd079d82fbd2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:16.669238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9740dbb9-4bf4-438e-a366-bd079d82fbd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:16.736743 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740dbb9-4bf4-438e-a366-bd079d82fbd2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:16.741158 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fdf09-81ae-4c31-bee4-e4a96c7da6c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:16.741780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fdf09-81ae-4c31-bee4-e4a96c7da6c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:16.742061 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fdf09-81ae-4c31-bee4-e4a96c7da6c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:16.742338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fdf09-81ae-4c31-bee4-e4a96c7da6c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:16.753063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fdf09-81ae-4c31-bee4-e4a96c7da6c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:16.755058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fdf09-81ae-4c31-bee4-e4a96c7da6c3 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:16.768478 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1178/2326] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:35:17.248357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947294c-b368-4526-8714-4de8b39ee9c4 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:17.249958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7947294c-b368-4526-8714-4de8b39ee9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:17.272425 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947294c-b368-4526-8714-4de8b39ee9c4 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:17.277320 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da4e23-ab03-4416-bb2d-fcde06d77f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:17.278079 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da4e23-ab03-4416-bb2d-fcde06d77f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:17.278364 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da4e23-ab03-4416-bb2d-fcde06d77f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:17.278640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da4e23-ab03-4416-bb2d-fcde06d77f48 admin admin] PATH_INFO: `/v3/projects/8bdb939f77eb4cf0ac73c4aac8a42036` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:17.279796 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da4e23-ab03-4416-bb2d-fcde06d77f48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bdb939f77eb4cf0ac73c4aac8a42036)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:17.281701 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3da4e23-ab03-4416-bb2d-fcde06d77f48 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:17.321554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:17.332179 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a3da4e23-ab03-4416-bb2d-fcde06d77f48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bdb939f77eb4cf0ac73c4aac8a42036', 'request_id': 'req-a3da4e23-ab03-4416-bb2d-fcde06d77f48'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:17.338094 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a3da4e23-ab03-4416-bb2d-fcde06d77f48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bdb939f77eb4cf0ac73c4aac8a42036', 'request_id': 'req-a3da4e23-ab03-4416-bb2d-fcde06d77f48'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:17.349703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a3da4e23-ab03-4416-bb2d-fcde06d77f48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bdb939f77eb4cf0ac73c4aac8a42036', 'request_id': 'req-a3da4e23-ab03-4416-bb2d-fcde06d77f48'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:17.363835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a3da4e23-ab03-4416-bb2d-fcde06d77f48 admin admin] The token cache is being invalidate because project 8bdb939f77eb4cf0ac73c4aac8a42036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:35:17.364134 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a3da4e23-ab03-4416-bb2d-fcde06d77f48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:17.364836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a3da4e23-ab03-4416-bb2d-fcde06d77f48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:35:17.375728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1149/2327] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:35:17 2022] DELETE /identity/v3/projects/8bdb939f77eb4cf0ac73c4aac8a42036 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:35:18.619326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516f6d9-0d80-4166-89bd-c305f5488736 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:18.619803 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516f6d9-0d80-4166-89bd-c305f5488736 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:18.619907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516f6d9-0d80-4166-89bd-c305f5488736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:18.639494 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e516f6d9-0d80-4166-89bd-c305f5488736 None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:18.640716 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e516f6d9-0d80-4166-89bd-c305f5488736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:18.694671 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1179/2328] 10.222.0.58 () {58 vars in 1009 bytes} [Wed Aug 24 12:35:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:35:18.716518 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da4e23-ab03-4416-bb2d-fcde06d77f48 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:18.717928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a3da4e23-ab03-4416-bb2d-fcde06d77f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:18.785406 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da4e23-ab03-4416-bb2d-fcde06d77f48 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:18.790220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b1861-5a1c-4fbe-8e8b-ce35318f1ed1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:18.790822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b1861-5a1c-4fbe-8e8b-ce35318f1ed1 service swift] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:18.791100 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b1861-5a1c-4fbe-8e8b-ce35318f1ed1 service swift] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:18.791373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b1861-5a1c-4fbe-8e8b-ce35318f1ed1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:18.799848 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b1861-5a1c-4fbe-8e8b-ce35318f1ed1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:18.801428 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b1861-5a1c-4fbe-8e8b-ce35318f1ed1 service swift] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:18.809800 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1150/2329] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:35:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:35:28.688062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=54,keystone:DELETE=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:28.806133 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:DELETE=5,keystone:SELECT=43,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:32.325474 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516f6d9-0d80-4166-89bd-c305f5488736 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:32.327845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e516f6d9-0d80-4166-89bd-c305f5488736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:32.391850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516f6d9-0d80-4166-89bd-c305f5488736 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:32.395396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82288f13-c2f5-4f2d-ab1c-0b3c80bb21f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:32.396032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82288f13-c2f5-4f2d-ab1c-0b3c80bb21f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:32.396312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82288f13-c2f5-4f2d-ab1c-0b3c80bb21f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:32.396585 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82288f13-c2f5-4f2d-ab1c-0b3c80bb21f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:32.398324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-82288f13-c2f5-4f2d-ab1c-0b3c80bb21f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:32.462940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82288f13-c2f5-4f2d-ab1c-0b3c80bb21f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:32.464429 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82288f13-c2f5-4f2d-ab1c-0b3c80bb21f4 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:32.474876 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1180/2330] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:35:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:35:33.607345 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b1861-5a1c-4fbe-8e8b-ce35318f1ed1 service swift] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:33.608981 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-552b1861-5a1c-4fbe-8e8b-ce35318f1ed1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:33.683280 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b1861-5a1c-4fbe-8e8b-ce35318f1ed1 service swift] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:33.689248 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975cf283-7a33-4c50-921d-0991138d9cc6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ba45e3cff74962965b248f9f017f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:33.689851 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975cf283-7a33-4c50-921d-0991138d9cc6 service placement] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:33.690130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975cf283-7a33-4c50-921d-0991138d9cc6 service placement] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:33.690400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975cf283-7a33-4c50-921d-0991138d9cc6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:33.692131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-975cf283-7a33-4c50-921d-0991138d9cc6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:33.704370 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975cf283-7a33-4c50-921d-0991138d9cc6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:33.706008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975cf283-7a33-4c50-921d-0991138d9cc6 service placement] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:33.716558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1151/2331] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:35:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:35:42.470465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=50 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:42.640067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2955a-4645-42d6-bba5-d3e9c5d3e2e8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:42.640351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2955a-4645-42d6-bba5-d3e9c5d3e2e8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:42.640622 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2955a-4645-42d6-bba5-d3e9c5d3e2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:42.660207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7cc2955a-4645-42d6-bba5-d3e9c5d3e2e8 None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:35:42.661525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7cc2955a-4645-42d6-bba5-d3e9c5d3e2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:35:42.669393 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1181/2332] 10.222.0.58 () {58 vars in 967 bytes} [Wed Aug 24 12:35:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:35:42.681352 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975cf283-7a33-4c50-921d-0991138d9cc6 service placement] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:35:42.692987 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975cf283-7a33-4c50-921d-0991138d9cc6 service placement] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:35:42.697895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f892040-6ba0-4688-b975-23e14128e79b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:35:42.698485 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f892040-6ba0-4688-b975-23e14128e79b service swift] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:35:42.698826 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f892040-6ba0-4688-b975-23e14128e79b service swift] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:35:42.699357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f892040-6ba0-4688-b975-23e14128e79b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:35:42.713179 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f892040-6ba0-4688-b975-23e14128e79b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:35:42.715295 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f892040-6ba0-4688-b975-23e14128e79b service swift] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:35:42.724759 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1152/2333] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:35:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:35:52.654535 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:35:52.721273 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=38 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:36:07.627111 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b6d07-80e7-496a-8df8-3fa03772e0eb None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:07.627692 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b6d07-80e7-496a-8df8-3fa03772e0eb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:07.627692 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b6d07-80e7-496a-8df8-3fa03772e0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:07.646056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-dd0b6d07-80e7-496a-8df8-3fa03772e0eb None None] MFA Rules not processed for user `9aa753d1e2c4430498d5b28f0304952f`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:36:07.647249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-dd0b6d07-80e7-496a-8df8-3fa03772e0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:36:07.703611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1182/2334] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:36:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:36:17.696089 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=25 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:36:24.224105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f98fc6-8325-4595-9743-1de3ae029c57 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:24.224626 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f98fc6-8325-4595-9743-1de3ae029c57 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:24.224731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f98fc6-8325-4595-9743-1de3ae029c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:24.244120 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-86f98fc6-8325-4595-9743-1de3ae029c57 None None] MFA Rules not processed for user `9138ac21bbdb4577a7f43b5ab471d572`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:36:24.245481 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-86f98fc6-8325-4595-9743-1de3ae029c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:36:24.300337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1153/2335] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:36:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:36:34.156253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fb557-c784-4700-b4d8-5a767e132a02 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:36:34.156821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fb557-c784-4700-b4d8-5a767e132a02 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:36:34.156907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173fb557-c784-4700-b4d8-5a767e132a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:36:34.175987 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-173fb557-c784-4700-b4d8-5a767e132a02 None None] MFA Rules not processed for user `28dac3ee2d5c4ff5ba2d15b59e57d2a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:36:34.177302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-173fb557-c784-4700-b4d8-5a767e132a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:36:34.237542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1183/2336] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:36:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:36:34.293262 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=25 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:36:44.230614 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=25 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:19.029998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa785ae6-65a2-40d1-ba90-350a618dac92 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:19.030559 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa785ae6-65a2-40d1-ba90-350a618dac92 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:19.030559 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa785ae6-65a2-40d1-ba90-350a618dac92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:19.049090 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-fa785ae6-65a2-40d1-ba90-350a618dac92 None None] MFA Rules not processed for user `9aa753d1e2c4430498d5b28f0304952f`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:19.050308 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fa785ae6-65a2-40d1-ba90-350a618dac92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:19.057275 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1154/2337] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:22.089496 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173fb557-c784-4700-b4d8-5a767e132a02 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:22.099709 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173fb557-c784-4700-b4d8-5a767e132a02 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:22.103271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc7141-f4e3-453a-9c96-4173b2b8c785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:22.103839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc7141-f4e3-453a-9c96-4173b2b8c785 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:22.104117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc7141-f4e3-453a-9c96-4173b2b8c785 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:22.104387 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc7141-f4e3-453a-9c96-4173b2b8c785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:22.106132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-84cc7141-f4e3-453a-9c96-4173b2b8c785 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:22.175429 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc7141-f4e3-453a-9c96-4173b2b8c785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:22.176920 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc7141-f4e3-453a-9c96-4173b2b8c785 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:22.187903 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1184/2338] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:25.225723 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa785ae6-65a2-40d1-ba90-350a618dac92 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:25.236647 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa785ae6-65a2-40d1-ba90-350a618dac92 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:25.240444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5363462-85a9-4210-86b4-e985d12e4fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:25.241136 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5363462-85a9-4210-86b4-e985d12e4fda service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:25.241489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5363462-85a9-4210-86b4-e985d12e4fda service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:25.241832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5363462-85a9-4210-86b4-e985d12e4fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:25.244166 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f5363462-85a9-4210-86b4-e985d12e4fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:25.260055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5363462-85a9-4210-86b4-e985d12e4fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:25.262152 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5363462-85a9-4210-86b4-e985d12e4fda service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:25.274245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1155/2339] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:27.817361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee213b-1997-4fcc-9f85-541d6b032fdf None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:27.817870 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee213b-1997-4fcc-9f85-541d6b032fdf None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:27.817992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee213b-1997-4fcc-9f85-541d6b032fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:27.837318 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c7ee213b-1997-4fcc-9f85-541d6b032fdf None None] MFA Rules not processed for user `ed0d97f3f08f46af851492aee6a8d0ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:27.838799 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c7ee213b-1997-4fcc-9f85-541d6b032fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:27.910932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1185/2340] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:30.364934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5363462-85a9-4210-86b4-e985d12e4fda service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:30.366532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f5363462-85a9-4210-86b4-e985d12e4fda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:30.378352 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5363462-85a9-4210-86b4-e985d12e4fda service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:30.382266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54407ebb-101e-405d-a718-d99256fd0e3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:30.382868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54407ebb-101e-405d-a718-d99256fd0e3f service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:30.383148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54407ebb-101e-405d-a718-d99256fd0e3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:30.383419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54407ebb-101e-405d-a718-d99256fd0e3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:30.385134 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-54407ebb-101e-405d-a718-d99256fd0e3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:30.453243 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54407ebb-101e-405d-a718-d99256fd0e3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:30.454820 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54407ebb-101e-405d-a718-d99256fd0e3f service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:30.464947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1156/2341] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:34.299839 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee213b-1997-4fcc-9f85-541d6b032fdf None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:34.301405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c7ee213b-1997-4fcc-9f85-541d6b032fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:34.318367 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee213b-1997-4fcc-9f85-541d6b032fdf None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:34.322959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf0ef5-b9ff-4329-acc6-dbc9ce439203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:34.323586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf0ef5-b9ff-4329-acc6-dbc9ce439203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:34.324557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf0ef5-b9ff-4329-acc6-dbc9ce439203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:34.324907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf0ef5-b9ff-4329-acc6-dbc9ce439203 admin admin] PATH_INFO: `/v3/users/9138ac21bbdb4577a7f43b5ab471d572` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:34.326597 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf0ef5-b9ff-4329-acc6-dbc9ce439203 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9138ac21bbdb4577a7f43b5ab471d572)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:34.328686 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bf0ef5-b9ff-4329-acc6-dbc9ce439203 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:34.377894 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-57bf0ef5-b9ff-4329-acc6-dbc9ce439203 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9138ac21bbdb4577a7f43b5ab471d572'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:34.378566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-57bf0ef5-b9ff-4329-acc6-dbc9ce439203 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9138ac21bbdb4577a7f43b5ab471d572'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:34.385637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-57bf0ef5-b9ff-4329-acc6-dbc9ce439203 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9138ac21bbdb4577a7f43b5ab471d572'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:34.394277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-57bf0ef5-b9ff-4329-acc6-dbc9ce439203 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9138ac21bbdb4577a7f43b5ab471d572'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:34.406629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-57bf0ef5-b9ff-4329-acc6-dbc9ce439203 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9138ac21bbdb4577a7f43b5ab471d572'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:34.415859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1186/2342] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:37:34 2022] DELETE /identity/v3/users/9138ac21bbdb4577a7f43b5ab471d572 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:34.591357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54407ebb-101e-405d-a718-d99256fd0e3f service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:34.601961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54407ebb-101e-405d-a718-d99256fd0e3f service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:34.668982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78bd80-f27b-48d7-8984-432079ef3881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:34.669616 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78bd80-f27b-48d7-8984-432079ef3881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:34.669958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78bd80-f27b-48d7-8984-432079ef3881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:34.670233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78bd80-f27b-48d7-8984-432079ef3881 admin admin] PATH_INFO: `/v3/projects/9afa0216c9e140bb82f56f14ccfb7bb6` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:34.671335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78bd80-f27b-48d7-8984-432079ef3881 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9afa0216c9e140bb82f56f14ccfb7bb6)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:34.673269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78bd80-f27b-48d7-8984-432079ef3881 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:34.721698 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6e78bd80-f27b-48d7-8984-432079ef3881 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9afa0216c9e140bb82f56f14ccfb7bb6', 'request_id': 'req-6e78bd80-f27b-48d7-8984-432079ef3881'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:34.727153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6e78bd80-f27b-48d7-8984-432079ef3881 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9afa0216c9e140bb82f56f14ccfb7bb6', 'request_id': 'req-6e78bd80-f27b-48d7-8984-432079ef3881'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:34.737472 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6e78bd80-f27b-48d7-8984-432079ef3881 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9afa0216c9e140bb82f56f14ccfb7bb6', 'request_id': 'req-6e78bd80-f27b-48d7-8984-432079ef3881'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:34.750179 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6e78bd80-f27b-48d7-8984-432079ef3881 admin admin] The token cache is being invalidate because project 9afa0216c9e140bb82f56f14ccfb7bb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:37:34.750480 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6e78bd80-f27b-48d7-8984-432079ef3881 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:34.751144 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6e78bd80-f27b-48d7-8984-432079ef3881 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:34.752689 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1157/2343] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:37:34 2022] DELETE /identity/v3/projects/9afa0216c9e140bb82f56f14ccfb7bb6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:34.757774 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ef712-2091-4aaa-b995-d7e4c75bf169 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:34.758068 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ef712-2091-4aaa-b995-d7e4c75bf169 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:34.758348 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ef712-2091-4aaa-b995-d7e4c75bf169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:34.766306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:34.780895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-107ef712-2091-4aaa-b995-d7e4c75bf169 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:34.782131 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-107ef712-2091-4aaa-b995-d7e4c75bf169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:34.841848 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1187/2344] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:37:34.849615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c4d42-d906-46e9-b03d-1d8377505ec5 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:34.849909 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c4d42-d906-46e9-b03d-1d8377505ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:34.850189 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c4d42-d906-46e9-b03d-1d8377505ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:34.859482 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:34.872195 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-3d7c4d42-d906-46e9-b03d-1d8377505ec5 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:34.873757 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-3d7c4d42-d906-46e9-b03d-1d8377505ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:34.881635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1158/2345] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:34.885847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ef712-2091-4aaa-b995-d7e4c75bf169 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:34.896157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ef712-2091-4aaa-b995-d7e4c75bf169 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:34.900785 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943a3bc-25e6-4a4e-84eb-f7f041a0403f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:34.901381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943a3bc-25e6-4a4e-84eb-f7f041a0403f admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:34.901663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943a3bc-25e6-4a4e-84eb-f7f041a0403f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:34.902009 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943a3bc-25e6-4a4e-84eb-f7f041a0403f admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:34.902958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943a3bc-25e6-4a4e-84eb-f7f041a0403f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:37:34.903269 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943a3bc-25e6-4a4e-84eb-f7f041a0403f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:34.905201 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943a3bc-25e6-4a4e-84eb-f7f041a0403f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:34.919736 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1188/2346] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:37:34.924108 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c4d42-d906-46e9-b03d-1d8377505ec5 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:34.934099 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c4d42-d906-46e9-b03d-1d8377505ec5 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:34.938854 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd67b6a-c9f0-45c9-839c-1fc633869dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:34.939449 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd67b6a-c9f0-45c9-839c-1fc633869dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:34.939767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd67b6a-c9f0-45c9-839c-1fc633869dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:34.940042 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd67b6a-c9f0-45c9-839c-1fc633869dd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:34.940671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd67b6a-c9f0-45c9-839c-1fc633869dd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:34.942592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd67b6a-c9f0-45c9-839c-1fc633869dd0 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:34.970851 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1159/2347] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:37:34 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:37:34.974949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943a3bc-25e6-4a4e-84eb-f7f041a0403f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:34.985422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943a3bc-25e6-4a4e-84eb-f7f041a0403f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:35.061967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f01f1d-2978-429c-8b27-8b706b34f39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:35.062937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f01f1d-2978-429c-8b27-8b706b34f39a admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:35.063392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f01f1d-2978-429c-8b27-8b706b34f39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:35.063801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f01f1d-2978-429c-8b27-8b706b34f39a admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:35.064715 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f01f1d-2978-429c-8b27-8b706b34f39a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:35.066978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f01f1d-2978-429c-8b27-8b706b34f39a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:35.103109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1189/2348] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:37:34 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:37:35.104043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd67b6a-c9f0-45c9-839c-1fc633869dd0 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:35.115608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd67b6a-c9f0-45c9-839c-1fc633869dd0 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:35.121993 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f8aa6-fdfc-4235-99d6-dc6a5d48a23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:35.123022 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f8aa6-fdfc-4235-99d6-dc6a5d48a23c admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:35.123439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f8aa6-fdfc-4235-99d6-dc6a5d48a23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:35.123832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f8aa6-fdfc-4235-99d6-dc6a5d48a23c admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:35.125607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f8aa6-fdfc-4235-99d6-dc6a5d48a23c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:35.128051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246f8aa6-fdfc-4235-99d6-dc6a5d48a23c admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:35.142696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1160/2349] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:37:35.146650 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f01f1d-2978-429c-8b27-8b706b34f39a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:35.159906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f01f1d-2978-429c-8b27-8b706b34f39a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:35.165321 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47489fe5-2511-40d5-916b-4b4c96ae2daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:35.166113 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47489fe5-2511-40d5-916b-4b4c96ae2daf admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:35.166467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47489fe5-2511-40d5-916b-4b4c96ae2daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:35.166845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47489fe5-2511-40d5-916b-4b4c96ae2daf admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:35.168364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47489fe5-2511-40d5-916b-4b4c96ae2daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:35.170430 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47489fe5-2511-40d5-916b-4b4c96ae2daf admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:35.195862 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1190/2350] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:37:35.200034 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f8aa6-fdfc-4235-99d6-dc6a5d48a23c admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:35.215432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f8aa6-fdfc-4235-99d6-dc6a5d48a23c admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:35.223063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f498384-535e-4805-9fab-6f7cbd651a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:35.224070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f498384-535e-4805-9fab-6f7cbd651a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:35.224315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f498384-535e-4805-9fab-6f7cbd651a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:35.224594 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f498384-535e-4805-9fab-6f7cbd651a2b admin admin] PATH_INFO: `/v3/projects/b1bfc73cb4574b0681f668107c68b848/users/9bc446e5db664322afe837f185cb2bd6/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:35.244542 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f498384-535e-4805-9fab-6f7cbd651a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1bfc73cb4574b0681f668107c68b848, user_id=9bc446e5db664322afe837f185cb2bd6, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:35.246928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f498384-535e-4805-9fab-6f7cbd651a2b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:35.267222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1161/2351] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:37:35 2022] PUT /identity/v3/projects/b1bfc73cb4574b0681f668107c68b848/users/9bc446e5db664322afe837f185cb2bd6/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:37:35.278670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957cab7-156f-4056-a595-1198babf024d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:35.278982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957cab7-156f-4056-a595-1198babf024d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:35.279194 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957cab7-156f-4056-a595-1198babf024d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:35.317197 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-5957cab7-156f-4056-a595-1198babf024d None None] MFA Rules not processed for user `9bc446e5db664322afe837f185cb2bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:35.318922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5957cab7-156f-4056-a595-1198babf024d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:35.389857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:35.390144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1191/2352] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:35.394773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f498384-535e-4805-9fab-6f7cbd651a2b admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:35.458626 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f498384-535e-4805-9fab-6f7cbd651a2b admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:35.463456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136e963-3336-467a-be3e-284d407014de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:35.464083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136e963-3336-467a-be3e-284d407014de admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:35.465797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136e963-3336-467a-be3e-284d407014de admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:35.466083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136e963-3336-467a-be3e-284d407014de admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:35.466720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136e963-3336-467a-be3e-284d407014de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:35.468661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5136e963-3336-467a-be3e-284d407014de admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:35.526875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1162/2353] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:37:35 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:37:35.535597 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957cab7-156f-4056-a595-1198babf024d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:35.549412 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957cab7-156f-4056-a595-1198babf024d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:35.630234 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107ee29-483a-47dc-a53d-94e40a839527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:35.630960 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107ee29-483a-47dc-a53d-94e40a839527 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:35.631601 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107ee29-483a-47dc-a53d-94e40a839527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:35.631960 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2107ee29-483a-47dc-a53d-94e40a839527 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:35.632664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107ee29-483a-47dc-a53d-94e40a839527 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:35.634642 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2107ee29-483a-47dc-a53d-94e40a839527 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:35.662212 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1192/2354] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:37:35 2022] POST /identity/v3/users => generated 299 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:37:35.666116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136e963-3336-467a-be3e-284d407014de admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:35.681849 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5136e963-3336-467a-be3e-284d407014de admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:35.686656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a93b2e-22a0-43ec-af85-b5f8fb5df89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:35.687340 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a93b2e-22a0-43ec-af85-b5f8fb5df89c admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:35.687717 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a93b2e-22a0-43ec-af85-b5f8fb5df89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:35.688057 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a93b2e-22a0-43ec-af85-b5f8fb5df89c admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:35.689047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a93b2e-22a0-43ec-af85-b5f8fb5df89c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:35.690727 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a93b2e-22a0-43ec-af85-b5f8fb5df89c admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:35.703503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1163/2355] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:37:35.707798 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107ee29-483a-47dc-a53d-94e40a839527 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:35.718410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2107ee29-483a-47dc-a53d-94e40a839527 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:35.723039 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f2c2d-6d16-4669-9f0a-83f280ea30a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:35.723708 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f2c2d-6d16-4669-9f0a-83f280ea30a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:35.723990 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f2c2d-6d16-4669-9f0a-83f280ea30a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:35.724265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f2c2d-6d16-4669-9f0a-83f280ea30a0 admin admin] PATH_INFO: `/v3/projects/c72caa6dde5a45fb878ebbaa86d2f6d9/users/40755cbd267a4808a84cd6a567c7689d/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:35.740154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f2c2d-6d16-4669-9f0a-83f280ea30a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c72caa6dde5a45fb878ebbaa86d2f6d9, user_id=40755cbd267a4808a84cd6a567c7689d, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:35.742744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f2c2d-6d16-4669-9f0a-83f280ea30a0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:35.756929 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1193/2356] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:37:35 2022] PUT /identity/v3/projects/c72caa6dde5a45fb878ebbaa86d2f6d9/users/40755cbd267a4808a84cd6a567c7689d/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:35.761020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a93b2e-22a0-43ec-af85-b5f8fb5df89c admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:35.771762 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a93b2e-22a0-43ec-af85-b5f8fb5df89c admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:35.829376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f114a9-c0c4-4472-8df1-920b59c54de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:35.829984 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f114a9-c0c4-4472-8df1-920b59c54de7 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:35.830272 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f114a9-c0c4-4472-8df1-920b59c54de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:35.830546 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f114a9-c0c4-4472-8df1-920b59c54de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:35.831185 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f114a9-c0c4-4472-8df1-920b59c54de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:35.832803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f114a9-c0c4-4472-8df1-920b59c54de7 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:35.843230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1164/2357] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:37:35.847464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f2c2d-6d16-4669-9f0a-83f280ea30a0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:35.858487 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f2c2d-6d16-4669-9f0a-83f280ea30a0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:35.863071 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b8434-209c-4523-8175-4d8d1e31da16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:35.863747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b8434-209c-4523-8175-4d8d1e31da16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:35.864032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b8434-209c-4523-8175-4d8d1e31da16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:35.864308 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b8434-209c-4523-8175-4d8d1e31da16 admin admin] PATH_INFO: `/v3/projects/c72caa6dde5a45fb878ebbaa86d2f6d9/users/40755cbd267a4808a84cd6a567c7689d/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:35.866244 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b8434-209c-4523-8175-4d8d1e31da16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c72caa6dde5a45fb878ebbaa86d2f6d9, user_id=40755cbd267a4808a84cd6a567c7689d, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:35.868912 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b8434-209c-4523-8175-4d8d1e31da16 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:35.888754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1194/2358] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:37:35 2022] PUT /identity/v3/projects/c72caa6dde5a45fb878ebbaa86d2f6d9/users/40755cbd267a4808a84cd6a567c7689d/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:35.900279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bfa8d-660f-4e20-a2b6-a43cdc16a721 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:35.900565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bfa8d-660f-4e20-a2b6-a43cdc16a721 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:35.900863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525bfa8d-660f-4e20-a2b6-a43cdc16a721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:35.922739 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:35.935038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-525bfa8d-660f-4e20-a2b6-a43cdc16a721 None None] MFA Rules not processed for user `40755cbd267a4808a84cd6a567c7689d`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:35.936286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-525bfa8d-660f-4e20-a2b6-a43cdc16a721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:36.005545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1165/2359] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:37:36.017491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b8434-209c-4523-8175-4d8d1e31da16 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:36.019296 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d57b8434-209c-4523-8175-4d8d1e31da16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:36.079943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b8434-209c-4523-8175-4d8d1e31da16 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:36.083958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42810008-20c4-4606-b152-75d42771236c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:36.084250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42810008-20c4-4606-b152-75d42771236c service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:36.084615 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42810008-20c4-4606-b152-75d42771236c service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:36.084802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42810008-20c4-4606-b152-75d42771236c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:36.094765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42810008-20c4-4606-b152-75d42771236c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:36.096249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42810008-20c4-4606-b152-75d42771236c service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:36.107336 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1195/2360] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:36.176154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bfa8d-660f-4e20-a2b6-a43cdc16a721 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:36.238392 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525bfa8d-660f-4e20-a2b6-a43cdc16a721 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:36.242904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d34ff-eb17-4ec2-b0df-0b80c5f500dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:36.243531 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d34ff-eb17-4ec2-b0df-0b80c5f500dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:36.243860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d34ff-eb17-4ec2-b0df-0b80c5f500dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:36.244119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d34ff-eb17-4ec2-b0df-0b80c5f500dd admin admin] PATH_INFO: `/v3/users/9bc446e5db664322afe837f185cb2bd6` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:36.245244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d34ff-eb17-4ec2-b0df-0b80c5f500dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bc446e5db664322afe837f185cb2bd6)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:36.247178 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d34ff-eb17-4ec2-b0df-0b80c5f500dd admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:36.293878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c48d34ff-eb17-4ec2-b0df-0b80c5f500dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bc446e5db664322afe837f185cb2bd6'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:36.294457 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c48d34ff-eb17-4ec2-b0df-0b80c5f500dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bc446e5db664322afe837f185cb2bd6'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:36.295054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c48d34ff-eb17-4ec2-b0df-0b80c5f500dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bc446e5db664322afe837f185cb2bd6'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:36.301942 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c48d34ff-eb17-4ec2-b0df-0b80c5f500dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bc446e5db664322afe837f185cb2bd6'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:36.309250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c48d34ff-eb17-4ec2-b0df-0b80c5f500dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bc446e5db664322afe837f185cb2bd6'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:36.328386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1166/2361] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:37:36 2022] DELETE /identity/v3/users/9bc446e5db664322afe837f185cb2bd6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:36.332585 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc37db5-cff7-4f1c-977b-cbd5a4698fee None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:36.333066 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc37db5-cff7-4f1c-977b-cbd5a4698fee None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:36.333177 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc37db5-cff7-4f1c-977b-cbd5a4698fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:36.353104 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-3bc37db5-cff7-4f1c-977b-cbd5a4698fee None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:36.354309 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3bc37db5-cff7-4f1c-977b-cbd5a4698fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:36.415046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:36.436637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1196/2362] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:36.443564 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d34ff-eb17-4ec2-b0df-0b80c5f500dd admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:36.445048 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c48d34ff-eb17-4ec2-b0df-0b80c5f500dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:36.468571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:36.512008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d34ff-eb17-4ec2-b0df-0b80c5f500dd admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:36.516233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e75d7-3e41-4991-bcdb-6b075e3e9b3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:36.516881 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e75d7-3e41-4991-bcdb-6b075e3e9b3d service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:36.517196 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e75d7-3e41-4991-bcdb-6b075e3e9b3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:36.517471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3e75d7-3e41-4991-bcdb-6b075e3e9b3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:36.527530 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e75d7-3e41-4991-bcdb-6b075e3e9b3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:36.529016 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3e75d7-3e41-4991-bcdb-6b075e3e9b3d service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:36.539261 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1167/2363] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:36.899211 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc37db5-cff7-4f1c-977b-cbd5a4698fee None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:36.900722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3bc37db5-cff7-4f1c-977b-cbd5a4698fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:36.915493 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc37db5-cff7-4f1c-977b-cbd5a4698fee None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:36.921372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:36.922011 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:36.922347 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:36.922625 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17 admin admin] PATH_INFO: `/v3/projects/b1bfc73cb4574b0681f668107c68b848` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:36.923834 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1bfc73cb4574b0681f668107c68b848)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:36.927044 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:36.979147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1bfc73cb4574b0681f668107c68b848', 'request_id': 'req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:36.988365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1bfc73cb4574b0681f668107c68b848', 'request_id': 'req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:36.998780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1bfc73cb4574b0681f668107c68b848', 'request_id': 'req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:37.012076 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17 admin admin] The token cache is being invalidate because project b1bfc73cb4574b0681f668107c68b848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:37:37.012378 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:37.013035 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:37.014606 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1197/2364] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:37:36 2022] DELETE /identity/v3/projects/b1bfc73cb4574b0681f668107c68b848 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:37.018646 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e75d7-3e41-4991-bcdb-6b075e3e9b3d service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:37.028567 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3e75d7-3e41-4991-bcdb-6b075e3e9b3d service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:37.089819 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a517a-db4c-4f3f-8dc3-1b1ca37a9293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:37.090463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a517a-db4c-4f3f-8dc3-1b1ca37a9293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:37.090746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a517a-db4c-4f3f-8dc3-1b1ca37a9293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:37.091047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a517a-db4c-4f3f-8dc3-1b1ca37a9293 admin admin] PATH_INFO: `/v3/users/40755cbd267a4808a84cd6a567c7689d` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:37.092196 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a517a-db4c-4f3f-8dc3-1b1ca37a9293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40755cbd267a4808a84cd6a567c7689d)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:37.094144 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a517a-db4c-4f3f-8dc3-1b1ca37a9293 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:37.149336 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c54a517a-db4c-4f3f-8dc3-1b1ca37a9293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40755cbd267a4808a84cd6a567c7689d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:37.150118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c54a517a-db4c-4f3f-8dc3-1b1ca37a9293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40755cbd267a4808a84cd6a567c7689d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:37.150923 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c54a517a-db4c-4f3f-8dc3-1b1ca37a9293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40755cbd267a4808a84cd6a567c7689d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:37.158578 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c54a517a-db4c-4f3f-8dc3-1b1ca37a9293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40755cbd267a4808a84cd6a567c7689d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:37.170893 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-c54a517a-db4c-4f3f-8dc3-1b1ca37a9293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40755cbd267a4808a84cd6a567c7689d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:37.197700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1168/2365] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:37:37 2022] DELETE /identity/v3/users/40755cbd267a4808a84cd6a567c7689d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:37.518495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:37.520105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:37.539224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb1017-b7d5-44b4-ac36-2c9b8f776f17 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:37.604365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:37.605288 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:37.605643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:37.606058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e admin admin] PATH_INFO: `/v3/projects/c72caa6dde5a45fb878ebbaa86d2f6d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:37.607405 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c72caa6dde5a45fb878ebbaa86d2f6d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:37.609211 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a517a-db4c-4f3f-8dc3-1b1ca37a9293 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:37.610821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c54a517a-db4c-4f3f-8dc3-1b1ca37a9293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:37.611042 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:37.659694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:37.671137 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c72caa6dde5a45fb878ebbaa86d2f6d9', 'request_id': 'req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:37.678823 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c72caa6dde5a45fb878ebbaa86d2f6d9', 'request_id': 'req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:37.691741 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c72caa6dde5a45fb878ebbaa86d2f6d9', 'request_id': 'req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:37.699789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a517a-db4c-4f3f-8dc3-1b1ca37a9293 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:37.707901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e admin admin] The token cache is being invalidate because project c72caa6dde5a45fb878ebbaa86d2f6d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:37:37.708773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:37.709497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:37.712396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1198/2366] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:37:37 2022] DELETE /identity/v3/projects/c72caa6dde5a45fb878ebbaa86d2f6d9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:37.712992 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:37.714392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:37.725685 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:37.774877 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1134c9-a5bd-46f6-997e-d91d65d82220 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:37.775497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1134c9-a5bd-46f6-997e-d91d65d82220 service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:37.775815 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1134c9-a5bd-46f6-997e-d91d65d82220 service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:37.776141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1134c9-a5bd-46f6-997e-d91d65d82220 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:37.778410 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-8c1134c9-a5bd-46f6-997e-d91d65d82220 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:37.806465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9d0a6-e25c-480a-aa3d-016364e3fa0e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:37.813950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9a5fe-91d0-44e8-a9d4-9b5931837ec2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:37.814576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9a5fe-91d0-44e8-a9d4-9b5931837ec2 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:37.814857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9a5fe-91d0-44e8-a9d4-9b5931837ec2 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:37.815133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9a5fe-91d0-44e8-a9d4-9b5931837ec2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:37.816897 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f5e9a5fe-91d0-44e8-a9d4-9b5931837ec2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:37.846914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1134c9-a5bd-46f6-997e-d91d65d82220 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:37.848571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1134c9-a5bd-46f6-997e-d91d65d82220 service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:37.859203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1169/2367] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:37:37.860566 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc4e10-16ef-4e85-b2aa-fae2b8e4008b None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:37.861183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc4e10-16ef-4e85-b2aa-fae2b8e4008b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:37.861183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc4e10-16ef-4e85-b2aa-fae2b8e4008b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:37.888704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-e6bc4e10-16ef-4e85-b2aa-fae2b8e4008b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:37.890605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e6bc4e10-16ef-4e85-b2aa-fae2b8e4008b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:37.901559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e9a5fe-91d0-44e8-a9d4-9b5931837ec2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:37.904181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e9a5fe-91d0-44e8-a9d4-9b5931837ec2 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:37.914780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1199/2368] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:37 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:37.970170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1170/2369] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:37.978042 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688eebc3-7831-42e2-a371-b2ba29a540d3 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:37.978331 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688eebc3-7831-42e2-a371-b2ba29a540d3 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:37.978601 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688eebc3-7831-42e2-a371-b2ba29a540d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:37.998794 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-688eebc3-7831-42e2-a371-b2ba29a540d3 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:38.000038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-688eebc3-7831-42e2-a371-b2ba29a540d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:38.008025 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1200/2370] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:38.013016 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc4e10-16ef-4e85-b2aa-fae2b8e4008b None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:38.023985 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc4e10-16ef-4e85-b2aa-fae2b8e4008b None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:38.029230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5181d-a0b5-426f-ae52-66ffe494fcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:38.029880 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5181d-a0b5-426f-ae52-66ffe494fcac admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:38.030153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5181d-a0b5-426f-ae52-66ffe494fcac admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:38.030430 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5181d-a0b5-426f-ae52-66ffe494fcac admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:38.031081 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5181d-a0b5-426f-ae52-66ffe494fcac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:37:38.031387 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5181d-a0b5-426f-ae52-66ffe494fcac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:38.033025 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5181d-a0b5-426f-ae52-66ffe494fcac admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:38.042014 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:38.045607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1171/2371] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:37:38.050025 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688eebc3-7831-42e2-a371-b2ba29a540d3 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:38.060567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688eebc3-7831-42e2-a371-b2ba29a540d3 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:38.065310 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507aa2ae-9582-4887-bd53-ae38dff30780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:38.065946 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aa2ae-9582-4887-bd53-ae38dff30780 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:38.066230 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aa2ae-9582-4887-bd53-ae38dff30780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:38.066534 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507aa2ae-9582-4887-bd53-ae38dff30780 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:38.067175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507aa2ae-9582-4887-bd53-ae38dff30780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:38.069138 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507aa2ae-9582-4887-bd53-ae38dff30780 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:38.102920 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1201/2372] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:37:38 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:37:38.104442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5181d-a0b5-426f-ae52-66ffe494fcac admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:38.115155 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5181d-a0b5-426f-ae52-66ffe494fcac admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:38.178006 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5055c21-e2f1-4304-a55c-a02e7d4e302f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:38.178629 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5055c21-e2f1-4304-a55c-a02e7d4e302f admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:38.178920 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5055c21-e2f1-4304-a55c-a02e7d4e302f admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:38.179199 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5055c21-e2f1-4304-a55c-a02e7d4e302f admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:38.179875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5055c21-e2f1-4304-a55c-a02e7d4e302f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:38.181832 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5055c21-e2f1-4304-a55c-a02e7d4e302f admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:38.205470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1172/2373] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:37:38 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:37:38.209708 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507aa2ae-9582-4887-bd53-ae38dff30780 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:38.220863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507aa2ae-9582-4887-bd53-ae38dff30780 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:38.225944 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb50817-b1d1-4b0a-9298-1022ddaa3eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:38.226565 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb50817-b1d1-4b0a-9298-1022ddaa3eef admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:38.226846 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb50817-b1d1-4b0a-9298-1022ddaa3eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:38.227121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb50817-b1d1-4b0a-9298-1022ddaa3eef admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:38.227804 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb50817-b1d1-4b0a-9298-1022ddaa3eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:38.229431 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb50817-b1d1-4b0a-9298-1022ddaa3eef admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:38.244017 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1202/2374] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:37:38.245478 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5055c21-e2f1-4304-a55c-a02e7d4e302f admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:38.256035 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5055c21-e2f1-4304-a55c-a02e7d4e302f admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:38.260902 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f333dd-388b-4218-a483-a99c79436d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:38.261786 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f333dd-388b-4218-a483-a99c79436d4d admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:38.262076 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f333dd-388b-4218-a483-a99c79436d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:38.262348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f333dd-388b-4218-a483-a99c79436d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:38.263005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f333dd-388b-4218-a483-a99c79436d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:38.264678 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f333dd-388b-4218-a483-a99c79436d4d admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:38.282400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1173/2375] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:37:38.283651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb50817-b1d1-4b0a-9298-1022ddaa3eef admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:38.295023 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb50817-b1d1-4b0a-9298-1022ddaa3eef admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:38.299671 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cac7be-8a2b-4bb5-b02d-f21dbdbfc62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:38.300338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cac7be-8a2b-4bb5-b02d-f21dbdbfc62d admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:38.300748 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cac7be-8a2b-4bb5-b02d-f21dbdbfc62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:38.301032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cac7be-8a2b-4bb5-b02d-f21dbdbfc62d admin admin] PATH_INFO: `/v3/projects/72eb01c64e3c44e7897ae171b364b47d/users/8a9ac61ee7314f0889118e0028df21c1/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:38.316343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cac7be-8a2b-4bb5-b02d-f21dbdbfc62d admin admin] RBAC: Authorizing `identity:create_grant(project_id=72eb01c64e3c44e7897ae171b364b47d, user_id=8a9ac61ee7314f0889118e0028df21c1, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:38.319174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cac7be-8a2b-4bb5-b02d-f21dbdbfc62d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:38.334375 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1203/2376] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:37:38 2022] PUT /identity/v3/projects/72eb01c64e3c44e7897ae171b364b47d/users/8a9ac61ee7314f0889118e0028df21c1/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:38.345231 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86174980-2056-4ecc-b897-df6db786d479 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:38.345523 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86174980-2056-4ecc-b897-df6db786d479 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:38.345797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86174980-2056-4ecc-b897-df6db786d479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:38.379177 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-86174980-2056-4ecc-b897-df6db786d479 None None] MFA Rules not processed for user `8a9ac61ee7314f0889118e0028df21c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:38.380787 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-86174980-2056-4ecc-b897-df6db786d479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:38.453781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1174/2377] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:38.465728 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cac7be-8a2b-4bb5-b02d-f21dbdbfc62d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:38.467481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-79cac7be-8a2b-4bb5-b02d-f21dbdbfc62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:38.519423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:38.539884 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cac7be-8a2b-4bb5-b02d-f21dbdbfc62d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:38.544505 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a0297-ff40-4bc1-9e55-a17e58f19c47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:38.545193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a0297-ff40-4bc1-9e55-a17e58f19c47 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:38.545486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a0297-ff40-4bc1-9e55-a17e58f19c47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:38.545813 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a0297-ff40-4bc1-9e55-a17e58f19c47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:38.554593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a0297-ff40-4bc1-9e55-a17e58f19c47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:38.556080 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a0297-ff40-4bc1-9e55-a17e58f19c47 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:38.566012 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1204/2378] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:38.590015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86174980-2056-4ecc-b897-df6db786d479 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:38.652717 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86174980-2056-4ecc-b897-df6db786d479 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:38.657054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b81d79-b5e3-4420-8e6a-469068610151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:38.657683 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b81d79-b5e3-4420-8e6a-469068610151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:38.657959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b81d79-b5e3-4420-8e6a-469068610151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:38.658229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b81d79-b5e3-4420-8e6a-469068610151 admin admin] PATH_INFO: `/v3/users/8a9ac61ee7314f0889118e0028df21c1` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:38.659373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b81d79-b5e3-4420-8e6a-469068610151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a9ac61ee7314f0889118e0028df21c1)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:38.661326 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b81d79-b5e3-4420-8e6a-469068610151 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:38.672959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:38.715775 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-37b81d79-b5e3-4420-8e6a-469068610151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a9ac61ee7314f0889118e0028df21c1'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:38.716418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-37b81d79-b5e3-4420-8e6a-469068610151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a9ac61ee7314f0889118e0028df21c1'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:38.717026 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-37b81d79-b5e3-4420-8e6a-469068610151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a9ac61ee7314f0889118e0028df21c1'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:38.724664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-37b81d79-b5e3-4420-8e6a-469068610151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a9ac61ee7314f0889118e0028df21c1'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:38.732302 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-37b81d79-b5e3-4420-8e6a-469068610151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a9ac61ee7314f0889118e0028df21c1'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:38.750518 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1175/2379] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:37:38 2022] DELETE /identity/v3/users/8a9ac61ee7314f0889118e0028df21c1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:38.754774 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50a4d5-4baa-43ac-95d1-de2ee9e71512 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:38.755127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50a4d5-4baa-43ac-95d1-de2ee9e71512 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:38.755482 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50a4d5-4baa-43ac-95d1-de2ee9e71512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:38.774495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1d50a4d5-4baa-43ac-95d1-de2ee9e71512 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:38.775721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1d50a4d5-4baa-43ac-95d1-de2ee9e71512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:38.841362 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1205/2380] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:38.851457 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b81d79-b5e3-4420-8e6a-469068610151 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:38.853032 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-37b81d79-b5e3-4420-8e6a-469068610151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:38.922030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b81d79-b5e3-4420-8e6a-469068610151 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:38.925751 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52488514-e40f-40f9-b20b-03c7a0af9318 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:38.926229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52488514-e40f-40f9-b20b-03c7a0af9318 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:38.926518 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52488514-e40f-40f9-b20b-03c7a0af9318 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:38.926791 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52488514-e40f-40f9-b20b-03c7a0af9318 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:38.938636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52488514-e40f-40f9-b20b-03c7a0af9318 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:38.940200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52488514-e40f-40f9-b20b-03c7a0af9318 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:38.951093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1176/2381] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:39.298640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50a4d5-4baa-43ac-95d1-de2ee9e71512 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:39.299775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1d50a4d5-4baa-43ac-95d1-de2ee9e71512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:39.315760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50a4d5-4baa-43ac-95d1-de2ee9e71512 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:39.321597 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968f9f2-ed37-41e6-9e76-9c6e208a8989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:39.322565 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968f9f2-ed37-41e6-9e76-9c6e208a8989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:39.323299 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968f9f2-ed37-41e6-9e76-9c6e208a8989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:39.324395 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968f9f2-ed37-41e6-9e76-9c6e208a8989 admin admin] PATH_INFO: `/v3/projects/72eb01c64e3c44e7897ae171b364b47d` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:39.326799 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968f9f2-ed37-41e6-9e76-9c6e208a8989 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72eb01c64e3c44e7897ae171b364b47d)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:39.329156 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968f9f2-ed37-41e6-9e76-9c6e208a8989 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:39.393079 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9968f9f2-ed37-41e6-9e76-9c6e208a8989 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72eb01c64e3c44e7897ae171b364b47d', 'request_id': 'req-9968f9f2-ed37-41e6-9e76-9c6e208a8989'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:39.405944 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9968f9f2-ed37-41e6-9e76-9c6e208a8989 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72eb01c64e3c44e7897ae171b364b47d', 'request_id': 'req-9968f9f2-ed37-41e6-9e76-9c6e208a8989'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:39.415849 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9968f9f2-ed37-41e6-9e76-9c6e208a8989 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72eb01c64e3c44e7897ae171b364b47d', 'request_id': 'req-9968f9f2-ed37-41e6-9e76-9c6e208a8989'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:39.430572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9968f9f2-ed37-41e6-9e76-9c6e208a8989 admin admin] The token cache is being invalidate because project 72eb01c64e3c44e7897ae171b364b47d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:37:39.430871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9968f9f2-ed37-41e6-9e76-9c6e208a8989 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:39.431723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-9968f9f2-ed37-41e6-9e76-9c6e208a8989 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:39.434096 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1206/2382] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:37:39 2022] DELETE /identity/v3/projects/72eb01c64e3c44e7897ae171b364b47d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:39.439058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ce2ac-0bf0-43f8-9ba0-9850d44546b5 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:39.439552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ce2ac-0bf0-43f8-9ba0-9850d44546b5 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:39.439685 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ce2ac-0bf0-43f8-9ba0-9850d44546b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:39.463073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-f81ce2ac-0bf0-43f8-9ba0-9850d44546b5 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:39.464962 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f81ce2ac-0bf0-43f8-9ba0-9850d44546b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:39.563053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1177/2383] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:39.572715 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afa765-a4b0-4b27-b908-a97bd2d4e7fb None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:39.573277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afa765-a4b0-4b27-b908-a97bd2d4e7fb None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:39.573277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afa765-a4b0-4b27-b908-a97bd2d4e7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:39.597049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e2afa765-a4b0-4b27-b908-a97bd2d4e7fb None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:39.598963 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e2afa765-a4b0-4b27-b908-a97bd2d4e7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:39.612651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1207/2384] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:39.617024 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ce2ac-0bf0-43f8-9ba0-9850d44546b5 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:39.629337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ce2ac-0bf0-43f8-9ba0-9850d44546b5 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:39.635084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd527c-9edd-4917-91b4-436b49ba8cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:39.636453 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd527c-9edd-4917-91b4-436b49ba8cad admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:39.636898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd527c-9edd-4917-91b4-436b49ba8cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:39.637269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd527c-9edd-4917-91b4-436b49ba8cad admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:39.637868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd527c-9edd-4917-91b4-436b49ba8cad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:37:39.638177 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd527c-9edd-4917-91b4-436b49ba8cad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:39.640053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd527c-9edd-4917-91b4-436b49ba8cad admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:39.655155 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1178/2385] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:37:39.659265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afa765-a4b0-4b27-b908-a97bd2d4e7fb None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:39.669744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afa765-a4b0-4b27-b908-a97bd2d4e7fb None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:39.675381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bbdaa-9c13-4fa7-bf9a-9337d1c3718c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:39.676101 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bbdaa-9c13-4fa7-bf9a-9337d1c3718c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:39.676393 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bbdaa-9c13-4fa7-bf9a-9337d1c3718c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:39.677372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bbdaa-9c13-4fa7-bf9a-9337d1c3718c admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:39.677372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bbdaa-9c13-4fa7-bf9a-9337d1c3718c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:39.679218 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bbdaa-9c13-4fa7-bf9a-9337d1c3718c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:39.717851 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1208/2386] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:37:39 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:37:39.719123 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd527c-9edd-4917-91b4-436b49ba8cad admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:39.730809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd527c-9edd-4917-91b4-436b49ba8cad admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:39.746605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:39.816055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2e60b-a0f6-4f5c-ad54-59c4774e2b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:39.816670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2e60b-a0f6-4f5c-ad54-59c4774e2b0f admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:39.816956 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2e60b-a0f6-4f5c-ad54-59c4774e2b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:39.817231 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2e60b-a0f6-4f5c-ad54-59c4774e2b0f admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:39.817921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2e60b-a0f6-4f5c-ad54-59c4774e2b0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:39.820051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2e60b-a0f6-4f5c-ad54-59c4774e2b0f admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:39.846545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1179/2387] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:37:39 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:37:39.850614 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bbdaa-9c13-4fa7-bf9a-9337d1c3718c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:39.865047 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bbdaa-9c13-4fa7-bf9a-9337d1c3718c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:39.870146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238064af-9500-4622-9b51-89a7ffcc6302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:39.871837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238064af-9500-4622-9b51-89a7ffcc6302 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:39.872298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238064af-9500-4622-9b51-89a7ffcc6302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:39.872862 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238064af-9500-4622-9b51-89a7ffcc6302 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:39.874616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238064af-9500-4622-9b51-89a7ffcc6302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:39.876641 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238064af-9500-4622-9b51-89a7ffcc6302 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:39.892508 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1209/2388] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:37:39.893908 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2e60b-a0f6-4f5c-ad54-59c4774e2b0f admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:39.904391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2e60b-a0f6-4f5c-ad54-59c4774e2b0f admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:39.908919 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02091a-40f2-40bc-9da8-da4c852dd3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:39.909524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02091a-40f2-40bc-9da8-da4c852dd3ab admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:39.909804 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02091a-40f2-40bc-9da8-da4c852dd3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:39.910076 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02091a-40f2-40bc-9da8-da4c852dd3ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:39.910720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02091a-40f2-40bc-9da8-da4c852dd3ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:39.912363 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02091a-40f2-40bc-9da8-da4c852dd3ab admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:39.923416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1180/2389] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:37:39.927616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238064af-9500-4622-9b51-89a7ffcc6302 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:39.939163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238064af-9500-4622-9b51-89a7ffcc6302 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:39.944132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bf42a-e492-452b-8810-c884f1c0d559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:39.944790 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bf42a-e492-452b-8810-c884f1c0d559 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:39.945098 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bf42a-e492-452b-8810-c884f1c0d559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:39.945371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bf42a-e492-452b-8810-c884f1c0d559 admin admin] PATH_INFO: `/v3/projects/f888a72dd9374ee6900c0f31d67c1b14/users/3690bffd461b4269993af0204d754b65/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:39.961547 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144bf42a-e492-452b-8810-c884f1c0d559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f888a72dd9374ee6900c0f31d67c1b14, user_id=3690bffd461b4269993af0204d754b65, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:39.964431 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144bf42a-e492-452b-8810-c884f1c0d559 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:39.979165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1210/2390] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:37:39 2022] PUT /identity/v3/projects/f888a72dd9374ee6900c0f31d67c1b14/users/3690bffd461b4269993af0204d754b65/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:39.990249 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4686b2-936e-4bee-8680-a3f4b5845034 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:39.990532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4686b2-936e-4bee-8680-a3f4b5845034 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:39.990816 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4686b2-936e-4bee-8680-a3f4b5845034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:40.019535 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-df4686b2-936e-4bee-8680-a3f4b5845034 None None] MFA Rules not processed for user `3690bffd461b4269993af0204d754b65`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:40.020850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-df4686b2-936e-4bee-8680-a3f4b5845034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:40.089751 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1181/2391] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:40.100536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bf42a-e492-452b-8810-c884f1c0d559 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:40.102321 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-144bf42a-e492-452b-8810-c884f1c0d559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:40.108908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:40.164470 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144bf42a-e492-452b-8810-c884f1c0d559 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:40.168795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4217f7-326b-4193-9890-5d14483f350a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd8df2f3357d4660bc22366f9746f89c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:40.169407 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4217f7-326b-4193-9890-5d14483f350a service glance] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:40.169687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4217f7-326b-4193-9890-5d14483f350a service glance] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:40.169963 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4217f7-326b-4193-9890-5d14483f350a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:40.178632 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4217f7-326b-4193-9890-5d14483f350a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:40.180031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4217f7-326b-4193-9890-5d14483f350a service glance] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:40.189871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1211/2392] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:46.988726 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101fa9c-62cc-498a-a64f-f453117958b0 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:46.989316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101fa9c-62cc-498a-a64f-f453117958b0 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:46.989316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101fa9c-62cc-498a-a64f-f453117958b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:47.014229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-6101fa9c-62cc-498a-a64f-f453117958b0 None None] MFA Rules not processed for user `18f8f419b523440f8a4ca06dc83bff40`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:47.016179 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-6101fa9c-62cc-498a-a64f-f453117958b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:47.096766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1182/2393] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:47.918117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4217f7-326b-4193-9890-5d14483f350a service glance] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:47.919932 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8d4217f7-326b-4193-9890-5d14483f350a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:47.995908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4217f7-326b-4193-9890-5d14483f350a service glance] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:48.001716 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb631bd-df44-44af-b59f-4e5c50661cce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ba45e3cff74962965b248f9f017f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:48.002346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb631bd-df44-44af-b59f-4e5c50661cce service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:48.002626 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb631bd-df44-44af-b59f-4e5c50661cce service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:48.002901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb631bd-df44-44af-b59f-4e5c50661cce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:48.004632 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ceb631bd-df44-44af-b59f-4e5c50661cce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:48.017129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb631bd-df44-44af-b59f-4e5c50661cce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:48.018733 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb631bd-df44-44af-b59f-4e5c50661cce service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:48.028905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1212/2394] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:50.783432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7afe71-0e3d-4d49-8c4d-9c2f42ff4d6f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:50.783887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7afe71-0e3d-4d49-8c4d-9c2f42ff4d6f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:50.784005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7afe71-0e3d-4d49-8c4d-9c2f42ff4d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:50.804373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-3a7afe71-0e3d-4d49-8c4d-9c2f42ff4d6f None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:50.805570 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-3a7afe71-0e3d-4d49-8c4d-9c2f42ff4d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:50.832524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:50.864715 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1183/2395] 10.222.0.58 () {58 vars in 1009 bytes} [Wed Aug 24 12:37:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:50.916328 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb631bd-df44-44af-b59f-4e5c50661cce service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:50.918029 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ceb631bd-df44-44af-b59f-4e5c50661cce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:50.997496 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb631bd-df44-44af-b59f-4e5c50661cce service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:51.003106 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27772162-57cc-4d0c-9365-de91ee7b5f0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:51.003749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27772162-57cc-4d0c-9365-de91ee7b5f0b service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:51.004029 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27772162-57cc-4d0c-9365-de91ee7b5f0b service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:51.004392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27772162-57cc-4d0c-9365-de91ee7b5f0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:51.017757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27772162-57cc-4d0c-9365-de91ee7b5f0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:51.019383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27772162-57cc-4d0c-9365-de91ee7b5f0b service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:51.029709 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1213/2396] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:37:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:37:51.590968 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7afe71-0e3d-4d49-8c4d-9c2f42ff4d6f None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:51.594024 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-3a7afe71-0e3d-4d49-8c4d-9c2f42ff4d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:51.704976 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7afe71-0e3d-4d49-8c4d-9c2f42ff4d6f None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:51.711488 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67cfc3-b5ef-42c1-aa48-8a505bdeb335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:51.712280 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67cfc3-b5ef-42c1-aa48-8a505bdeb335 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:51.712589 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67cfc3-b5ef-42c1-aa48-8a505bdeb335 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:51.712839 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67cfc3-b5ef-42c1-aa48-8a505bdeb335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:51.714603 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-8a67cfc3-b5ef-42c1-aa48-8a505bdeb335 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:51.727857 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67cfc3-b5ef-42c1-aa48-8a505bdeb335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:51.729729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67cfc3-b5ef-42c1-aa48-8a505bdeb335 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:51.745476 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1184/2397] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:57.060666 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27772162-57cc-4d0c-9365-de91ee7b5f0b service swift] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:57.105891 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:57.152288 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27772162-57cc-4d0c-9365-de91ee7b5f0b service swift] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:57.157156 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea83ca9-6ffc-4051-a7a0-07ef04e6e434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:57.157846 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea83ca9-6ffc-4051-a7a0-07ef04e6e434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:57.158127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea83ca9-6ffc-4051-a7a0-07ef04e6e434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:57.158407 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea83ca9-6ffc-4051-a7a0-07ef04e6e434 admin admin] PATH_INFO: `/v3/users/3690bffd461b4269993af0204d754b65` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:57.159555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea83ca9-6ffc-4051-a7a0-07ef04e6e434 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3690bffd461b4269993af0204d754b65)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:57.161726 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea83ca9-6ffc-4051-a7a0-07ef04e6e434 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:57.245286 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1ea83ca9-6ffc-4051-a7a0-07ef04e6e434 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3690bffd461b4269993af0204d754b65'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:57.245940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1ea83ca9-6ffc-4051-a7a0-07ef04e6e434 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3690bffd461b4269993af0204d754b65'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:57.253980 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1ea83ca9-6ffc-4051-a7a0-07ef04e6e434 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3690bffd461b4269993af0204d754b65'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:57.262440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1ea83ca9-6ffc-4051-a7a0-07ef04e6e434 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3690bffd461b4269993af0204d754b65'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:57.280551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1ea83ca9-6ffc-4051-a7a0-07ef04e6e434 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3690bffd461b4269993af0204d754b65'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:57.288949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1214/2398] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:37:57 2022] DELETE /identity/v3/users/3690bffd461b4269993af0204d754b65 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:57.293425 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6a97a-cc36-4721-99b9-07d76ecab331 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:57.293941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6a97a-cc36-4721-99b9-07d76ecab331 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:57.293941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6a97a-cc36-4721-99b9-07d76ecab331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:57.313925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-dee6a97a-cc36-4721-99b9-07d76ecab331 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:57.315155 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-dee6a97a-cc36-4721-99b9-07d76ecab331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:57.390315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1185/2399] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:57.396998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea83ca9-6ffc-4051-a7a0-07ef04e6e434 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:57.398434 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1ea83ca9-6ffc-4051-a7a0-07ef04e6e434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:57.470350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea83ca9-6ffc-4051-a7a0-07ef04e6e434 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:57.477324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93df34e4-40f3-4d24-8a45-be53f632c97e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:57.478133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df34e4-40f3-4d24-8a45-be53f632c97e service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:57.478507 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df34e4-40f3-4d24-8a45-be53f632c97e service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:57.478891 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93df34e4-40f3-4d24-8a45-be53f632c97e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:57.491393 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df34e4-40f3-4d24-8a45-be53f632c97e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:57.492555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93df34e4-40f3-4d24-8a45-be53f632c97e service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:57.518276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1215/2400] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:37:57.900048 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6a97a-cc36-4721-99b9-07d76ecab331 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:57.901709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-dee6a97a-cc36-4721-99b9-07d76ecab331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:57.924362 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6a97a-cc36-4721-99b9-07d76ecab331 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:57.939386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd260d-c59b-41cf-bcc8-546f3c04e94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:57.940075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd260d-c59b-41cf-bcc8-546f3c04e94e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:57.940361 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd260d-c59b-41cf-bcc8-546f3c04e94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:57.940635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd260d-c59b-41cf-bcc8-546f3c04e94e admin admin] PATH_INFO: `/v3/projects/f888a72dd9374ee6900c0f31d67c1b14` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:57.941760 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd260d-c59b-41cf-bcc8-546f3c04e94e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f888a72dd9374ee6900c0f31d67c1b14)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:57.943852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd260d-c59b-41cf-bcc8-546f3c04e94e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:58.008216 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7cdd260d-c59b-41cf-bcc8-546f3c04e94e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f888a72dd9374ee6900c0f31d67c1b14', 'request_id': 'req-7cdd260d-c59b-41cf-bcc8-546f3c04e94e'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:58.015229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7cdd260d-c59b-41cf-bcc8-546f3c04e94e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f888a72dd9374ee6900c0f31d67c1b14', 'request_id': 'req-7cdd260d-c59b-41cf-bcc8-546f3c04e94e'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:58.024830 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7cdd260d-c59b-41cf-bcc8-546f3c04e94e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f888a72dd9374ee6900c0f31d67c1b14', 'request_id': 'req-7cdd260d-c59b-41cf-bcc8-546f3c04e94e'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:58.037779 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7cdd260d-c59b-41cf-bcc8-546f3c04e94e admin admin] The token cache is being invalidate because project f888a72dd9374ee6900c0f31d67c1b14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:37:58.038146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7cdd260d-c59b-41cf-bcc8-546f3c04e94e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:58.038768 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7cdd260d-c59b-41cf-bcc8-546f3c04e94e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:37:58.043755 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1186/2401] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:37:57 2022] DELETE /identity/v3/projects/f888a72dd9374ee6900c0f31d67c1b14 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:58.045587 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f1a73-a84b-428c-82a3-902d411eb6ab None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:58.045875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f1a73-a84b-428c-82a3-902d411eb6ab None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:58.046150 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f1a73-a84b-428c-82a3-902d411eb6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:58.069052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7a8f1a73-a84b-428c-82a3-902d411eb6ab None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:58.070603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7a8f1a73-a84b-428c-82a3-902d411eb6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:58.134524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:58.175846 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1216/2402] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:37:58.179702 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a844e8b-b23b-4ada-9021-b81f5b7bcfd7 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:58.179922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a844e8b-b23b-4ada-9021-b81f5b7bcfd7 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:58.180227 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a844e8b-b23b-4ada-9021-b81f5b7bcfd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:58.190909 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:58.214570 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-2a844e8b-b23b-4ada-9021-b81f5b7bcfd7 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:58.215855 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2a844e8b-b23b-4ada-9021-b81f5b7bcfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:58.229966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1187/2403] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:58.232907 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f1a73-a84b-428c-82a3-902d411eb6ab None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:58.248856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f1a73-a84b-428c-82a3-902d411eb6ab None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:58.256969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db91cd-73ec-47e1-9ee5-234b84818f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:58.257858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db91cd-73ec-47e1-9ee5-234b84818f5b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:58.258222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db91cd-73ec-47e1-9ee5-234b84818f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:58.258556 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db91cd-73ec-47e1-9ee5-234b84818f5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:58.259371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db91cd-73ec-47e1-9ee5-234b84818f5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:37:58.259822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db91cd-73ec-47e1-9ee5-234b84818f5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:58.262326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db91cd-73ec-47e1-9ee5-234b84818f5b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:58.282821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a844e8b-b23b-4ada-9021-b81f5b7bcfd7 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:58.287859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1217/2404] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:37:58.293998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a844e8b-b23b-4ada-9021-b81f5b7bcfd7 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:58.299167 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097325a3-dfdd-465e-a0c4-df6a58d7f923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:58.299819 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097325a3-dfdd-465e-a0c4-df6a58d7f923 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:58.300106 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097325a3-dfdd-465e-a0c4-df6a58d7f923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:58.300386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097325a3-dfdd-465e-a0c4-df6a58d7f923 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:58.301024 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097325a3-dfdd-465e-a0c4-df6a58d7f923 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:58.303094 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097325a3-dfdd-465e-a0c4-df6a58d7f923 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:58.347765 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1188/2405] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:37:58 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:37:58.351671 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db91cd-73ec-47e1-9ee5-234b84818f5b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:58.367584 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db91cd-73ec-47e1-9ee5-234b84818f5b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:58.444370 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38494ac-3900-42e5-9f46-61a6c3aa373c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:58.445334 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38494ac-3900-42e5-9f46-61a6c3aa373c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:58.445868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38494ac-3900-42e5-9f46-61a6c3aa373c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:58.446946 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38494ac-3900-42e5-9f46-61a6c3aa373c admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:58.447834 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38494ac-3900-42e5-9f46-61a6c3aa373c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:58.450119 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38494ac-3900-42e5-9f46-61a6c3aa373c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:58.479916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1218/2406] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:37:58 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:37:58.488098 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097325a3-dfdd-465e-a0c4-df6a58d7f923 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:58.500952 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097325a3-dfdd-465e-a0c4-df6a58d7f923 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:58.508728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc462e-52c1-4a75-a3da-79cb4537e18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:58.509415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc462e-52c1-4a75-a3da-79cb4537e18e admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:58.509783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc462e-52c1-4a75-a3da-79cb4537e18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:58.509958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc462e-52c1-4a75-a3da-79cb4537e18e admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:58.510639 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc462e-52c1-4a75-a3da-79cb4537e18e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:58.512308 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc462e-52c1-4a75-a3da-79cb4537e18e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:58.527791 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1189/2407] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:37:58.529576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38494ac-3900-42e5-9f46-61a6c3aa373c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:58.541566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38494ac-3900-42e5-9f46-61a6c3aa373c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:58.548592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a723c-b37f-4167-a757-42dde378c1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:58.549214 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a723c-b37f-4167-a757-42dde378c1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:58.549496 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a723c-b37f-4167-a757-42dde378c1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:58.549773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a723c-b37f-4167-a757-42dde378c1d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:58.550940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a723c-b37f-4167-a757-42dde378c1d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:58.554338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0a723c-b37f-4167-a757-42dde378c1d9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:58.576540 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1219/2408] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:37:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:37:58.577415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc462e-52c1-4a75-a3da-79cb4537e18e admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:58.588724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc462e-52c1-4a75-a3da-79cb4537e18e admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:58.593750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646bc9f-787d-4626-868a-0acf3d974803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:58.594423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646bc9f-787d-4626-868a-0acf3d974803 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:58.594700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646bc9f-787d-4626-868a-0acf3d974803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:58.594978 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646bc9f-787d-4626-868a-0acf3d974803 admin admin] PATH_INFO: `/v3/projects/02c993f281614c0bb85a3e262268bc2e/users/a7b14d7b62a542ce8ccc9a7d0a458239/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:58.610791 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646bc9f-787d-4626-868a-0acf3d974803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02c993f281614c0bb85a3e262268bc2e, user_id=a7b14d7b62a542ce8ccc9a7d0a458239, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:58.613545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646bc9f-787d-4626-868a-0acf3d974803 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:58.632006 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1190/2409] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:37:58 2022] PUT /identity/v3/projects/02c993f281614c0bb85a3e262268bc2e/users/a7b14d7b62a542ce8ccc9a7d0a458239/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:58.641791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df088e20-2909-464b-b983-fa628dfafd4e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:58.642219 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df088e20-2909-464b-b983-fa628dfafd4e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:58.642356 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df088e20-2909-464b-b983-fa628dfafd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:58.675399 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-df088e20-2909-464b-b983-fa628dfafd4e None None] MFA Rules not processed for user `a7b14d7b62a542ce8ccc9a7d0a458239`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:58.676664 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-df088e20-2909-464b-b983-fa628dfafd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:58.773327 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1220/2410] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:37:58.777899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646bc9f-787d-4626-868a-0acf3d974803 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:58.852376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646bc9f-787d-4626-868a-0acf3d974803 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:58.866503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b5efd-88e7-4df7-8a39-96bfe78d5721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:58.866643 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b5efd-88e7-4df7-8a39-96bfe78d5721 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:58.866853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b5efd-88e7-4df7-8a39-96bfe78d5721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:58.867131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b5efd-88e7-4df7-8a39-96bfe78d5721 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:58.867798 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b5efd-88e7-4df7-8a39-96bfe78d5721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:58.869817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b5efd-88e7-4df7-8a39-96bfe78d5721 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:58.917694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1191/2411] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:37:58 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:37:58.922052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df088e20-2909-464b-b983-fa628dfafd4e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:58.933602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df088e20-2909-464b-b983-fa628dfafd4e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:58.984481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:59.005482 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0dead-bdde-4491-833b-b32a07e93a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:59.006145 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0dead-bdde-4491-833b-b32a07e93a03 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:59.006428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0dead-bdde-4491-833b-b32a07e93a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:59.006698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc0dead-bdde-4491-833b-b32a07e93a03 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:59.007330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0dead-bdde-4491-833b-b32a07e93a03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:59.009311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc0dead-bdde-4491-833b-b32a07e93a03 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:59.035427 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1221/2412] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:37:58 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:37:59.036712 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b5efd-88e7-4df7-8a39-96bfe78d5721 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:59.051564 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b5efd-88e7-4df7-8a39-96bfe78d5721 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:59.058337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1305950-f870-4ef7-98ad-c62da8abb4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:59.058958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1305950-f870-4ef7-98ad-c62da8abb4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:59.059511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1305950-f870-4ef7-98ad-c62da8abb4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:59.059947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1305950-f870-4ef7-98ad-c62da8abb4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:59.060756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1305950-f870-4ef7-98ad-c62da8abb4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:59.062496 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1305950-f870-4ef7-98ad-c62da8abb4d9 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:59.079944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1192/2413] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:37:59.084285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0dead-bdde-4491-833b-b32a07e93a03 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:59.099591 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc0dead-bdde-4491-833b-b32a07e93a03 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:59.109061 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad9dcd-744c-4f0b-ba37-5c68ad1af699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:59.109800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad9dcd-744c-4f0b-ba37-5c68ad1af699 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:59.110085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad9dcd-744c-4f0b-ba37-5c68ad1af699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:59.110388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad9dcd-744c-4f0b-ba37-5c68ad1af699 admin admin] PATH_INFO: `/v3/projects/8b9085b947ed4c24998a9cff7205a168/users/97a7132a3be14df984e664d68b8ace32/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:59.132570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad9dcd-744c-4f0b-ba37-5c68ad1af699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b9085b947ed4c24998a9cff7205a168, user_id=97a7132a3be14df984e664d68b8ace32, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:59.135654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad9dcd-744c-4f0b-ba37-5c68ad1af699 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:59.159355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1222/2414] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:37:59 2022] PUT /identity/v3/projects/8b9085b947ed4c24998a9cff7205a168/users/97a7132a3be14df984e664d68b8ace32/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:59.164413 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1305950-f870-4ef7-98ad-c62da8abb4d9 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:59.182064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1305950-f870-4ef7-98ad-c62da8abb4d9 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:59.252949 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f2e13-454e-4412-b683-817e97cdd2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:59.254240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f2e13-454e-4412-b683-817e97cdd2c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:59.254240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f2e13-454e-4412-b683-817e97cdd2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:59.254240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f2e13-454e-4412-b683-817e97cdd2c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:59.254923 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f2e13-454e-4412-b683-817e97cdd2c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:59.256877 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f2e13-454e-4412-b683-817e97cdd2c9 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:59.272017 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1193/2415] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:37:59 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:37:59.273187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad9dcd-744c-4f0b-ba37-5c68ad1af699 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:59.285877 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad9dcd-744c-4f0b-ba37-5c68ad1af699 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:59.294022 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd56ff-2e6b-4753-ad3a-87771b602565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:59.294864 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd56ff-2e6b-4753-ad3a-87771b602565 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:59.295149 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd56ff-2e6b-4753-ad3a-87771b602565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:59.295423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd56ff-2e6b-4753-ad3a-87771b602565 admin admin] PATH_INFO: `/v3/projects/8b9085b947ed4c24998a9cff7205a168/users/97a7132a3be14df984e664d68b8ace32/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:59.297444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd56ff-2e6b-4753-ad3a-87771b602565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b9085b947ed4c24998a9cff7205a168, user_id=97a7132a3be14df984e664d68b8ace32, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:59.300534 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd56ff-2e6b-4753-ad3a-87771b602565 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:59.317699 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1223/2416] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:37:59 2022] PUT /identity/v3/projects/8b9085b947ed4c24998a9cff7205a168/users/97a7132a3be14df984e664d68b8ace32/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:37:59.330090 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5b989-5f51-4fb7-a041-b809137d741c None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:59.330787 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5b989-5f51-4fb7-a041-b809137d741c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:59.330896 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5b989-5f51-4fb7-a041-b809137d741c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:59.355660 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:37:59.366137 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-70a5b989-5f51-4fb7-a041-b809137d741c None None] MFA Rules not processed for user `97a7132a3be14df984e664d68b8ace32`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:37:59.367374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-70a5b989-5f51-4fb7-a041-b809137d741c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:59.456473 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1194/2417] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:37:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:37:59.468904 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd56ff-2e6b-4753-ad3a-87771b602565 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:37:59.470979 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-06dd56ff-2e6b-4753-ad3a-87771b602565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:37:59.571187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd56ff-2e6b-4753-ad3a-87771b602565 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:37:59.575363 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ad926-a340-4aef-98cf-80e5cdc255ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:37:59.576550 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ad926-a340-4aef-98cf-80e5cdc255ca service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:37:59.577048 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ad926-a340-4aef-98cf-80e5cdc255ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:37:59.577534 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ad926-a340-4aef-98cf-80e5cdc255ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:37:59.646178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ad926-a340-4aef-98cf-80e5cdc255ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:37:59.647834 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ad926-a340-4aef-98cf-80e5cdc255ca service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:37:59.664627 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1224/2418] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:37:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:02.336750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5b989-5f51-4fb7-a041-b809137d741c None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:02.402373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5b989-5f51-4fb7-a041-b809137d741c None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:02.407758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b1dcc-787b-4270-92cd-58d4e13e0bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:02.408414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b1dcc-787b-4270-92cd-58d4e13e0bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:02.408697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b1dcc-787b-4270-92cd-58d4e13e0bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:02.408981 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b1dcc-787b-4270-92cd-58d4e13e0bd1 admin admin] PATH_INFO: `/v3/users/a7b14d7b62a542ce8ccc9a7d0a458239` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:02.410127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b1dcc-787b-4270-92cd-58d4e13e0bd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7b14d7b62a542ce8ccc9a7d0a458239)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:02.412102 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b1dcc-787b-4270-92cd-58d4e13e0bd1 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:02.462159 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-740b1dcc-787b-4270-92cd-58d4e13e0bd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7b14d7b62a542ce8ccc9a7d0a458239'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:02.462895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-740b1dcc-787b-4270-92cd-58d4e13e0bd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7b14d7b62a542ce8ccc9a7d0a458239'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:02.463556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-740b1dcc-787b-4270-92cd-58d4e13e0bd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7b14d7b62a542ce8ccc9a7d0a458239'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:02.471582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-740b1dcc-787b-4270-92cd-58d4e13e0bd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7b14d7b62a542ce8ccc9a7d0a458239'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:02.480008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-740b1dcc-787b-4270-92cd-58d4e13e0bd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7b14d7b62a542ce8ccc9a7d0a458239'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:02.500526 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1195/2419] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:38:02 2022] DELETE /identity/v3/users/a7b14d7b62a542ce8ccc9a7d0a458239 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:02.505093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24ccb0-459d-4430-bde6-8c49548b6542 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:02.505582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24ccb0-459d-4430-bde6-8c49548b6542 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:02.505697 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24ccb0-459d-4430-bde6-8c49548b6542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:02.524240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-7e24ccb0-459d-4430-bde6-8c49548b6542 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:02.525448 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7e24ccb0-459d-4430-bde6-8c49548b6542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:02.573975 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:02.590107 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1225/2420] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:38:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:02.600150 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b1dcc-787b-4270-92cd-58d4e13e0bd1 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:02.602061 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-740b1dcc-787b-4270-92cd-58d4e13e0bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:02.618714 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:02.665818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b1dcc-787b-4270-92cd-58d4e13e0bd1 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:02.670234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850336d-8ea8-4e7c-af5a-9aef93c6b59b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:02.670878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850336d-8ea8-4e7c-af5a-9aef93c6b59b service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:02.671094 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850336d-8ea8-4e7c-af5a-9aef93c6b59b service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:02.671375 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850336d-8ea8-4e7c-af5a-9aef93c6b59b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:02.680648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e850336d-8ea8-4e7c-af5a-9aef93c6b59b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:02.682080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e850336d-8ea8-4e7c-af5a-9aef93c6b59b service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:02.692160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1196/2421] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:02.987070 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24ccb0-459d-4430-bde6-8c49548b6542 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:02.988948 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-7e24ccb0-459d-4430-bde6-8c49548b6542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:03.003001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24ccb0-459d-4430-bde6-8c49548b6542 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:03.020861 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:03.021792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:03.022492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:03.023193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3 admin admin] PATH_INFO: `/v3/projects/02c993f281614c0bb85a3e262268bc2e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:03.024522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02c993f281614c0bb85a3e262268bc2e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:03.026649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:03.086055 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02c993f281614c0bb85a3e262268bc2e', 'request_id': 'req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:03.093674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02c993f281614c0bb85a3e262268bc2e', 'request_id': 'req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:03.101979 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02c993f281614c0bb85a3e262268bc2e', 'request_id': 'req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:03.113831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3 admin admin] The token cache is being invalidate because project 02c993f281614c0bb85a3e262268bc2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:38:03.114130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:03.114777 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:03.116742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1226/2422] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:38:02 2022] DELETE /identity/v3/projects/02c993f281614c0bb85a3e262268bc2e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:03.120785 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850336d-8ea8-4e7c-af5a-9aef93c6b59b service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:03.131065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850336d-8ea8-4e7c-af5a-9aef93c6b59b service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:03.188701 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1acc36-f0be-48d2-9618-c59e888eed8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:03.189484 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1acc36-f0be-48d2-9618-c59e888eed8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:03.189791 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1acc36-f0be-48d2-9618-c59e888eed8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:03.190042 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1acc36-f0be-48d2-9618-c59e888eed8c admin admin] PATH_INFO: `/v3/users/97a7132a3be14df984e664d68b8ace32` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:03.191310 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1acc36-f0be-48d2-9618-c59e888eed8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=97a7132a3be14df984e664d68b8ace32)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:03.193316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1acc36-f0be-48d2-9618-c59e888eed8c admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:03.241335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1b1acc36-f0be-48d2-9618-c59e888eed8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97a7132a3be14df984e664d68b8ace32'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:03.241982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1b1acc36-f0be-48d2-9618-c59e888eed8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97a7132a3be14df984e664d68b8ace32'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:03.242575 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1b1acc36-f0be-48d2-9618-c59e888eed8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97a7132a3be14df984e664d68b8ace32'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:03.249858 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1b1acc36-f0be-48d2-9618-c59e888eed8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97a7132a3be14df984e664d68b8ace32'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:03.257866 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1b1acc36-f0be-48d2-9618-c59e888eed8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97a7132a3be14df984e664d68b8ace32'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:03.274846 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1197/2423] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:38:03 2022] DELETE /identity/v3/users/97a7132a3be14df984e664d68b8ace32 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:38:03.548305 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:03.549865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:03.563634 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b43d96-7231-4e7c-ad09-9eb974bc1dd3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:03.626654 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6ad20-b24e-4748-8241-cb032667efe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:03.627201 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6ad20-b24e-4748-8241-cb032667efe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:03.627482 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6ad20-b24e-4748-8241-cb032667efe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:03.627794 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6ad20-b24e-4748-8241-cb032667efe6 admin admin] PATH_INFO: `/v3/projects/8b9085b947ed4c24998a9cff7205a168` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:03.628938 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6ad20-b24e-4748-8241-cb032667efe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b9085b947ed4c24998a9cff7205a168)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:03.630844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6ad20-b24e-4748-8241-cb032667efe6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:03.681386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-24c6ad20-b24e-4748-8241-cb032667efe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b9085b947ed4c24998a9cff7205a168', 'request_id': 'req-24c6ad20-b24e-4748-8241-cb032667efe6'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:03.687570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-24c6ad20-b24e-4748-8241-cb032667efe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b9085b947ed4c24998a9cff7205a168', 'request_id': 'req-24c6ad20-b24e-4748-8241-cb032667efe6'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:03.695195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-24c6ad20-b24e-4748-8241-cb032667efe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b9085b947ed4c24998a9cff7205a168', 'request_id': 'req-24c6ad20-b24e-4748-8241-cb032667efe6'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:03.710905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-24c6ad20-b24e-4748-8241-cb032667efe6 admin admin] The token cache is being invalidate because project 8b9085b947ed4c24998a9cff7205a168 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:38:03.711181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-24c6ad20-b24e-4748-8241-cb032667efe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:03.711882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-24c6ad20-b24e-4748-8241-cb032667efe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:03.713598 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1227/2424] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:38:03 2022] DELETE /identity/v3/projects/8b9085b947ed4c24998a9cff7205a168 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:03.718688 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0caa3d-9723-48bb-8ae5-b0a2a4bedabf None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:03.719167 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0caa3d-9723-48bb-8ae5-b0a2a4bedabf None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:03.719353 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0caa3d-9723-48bb-8ae5-b0a2a4bedabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:03.739405 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-0f0caa3d-9723-48bb-8ae5-b0a2a4bedabf None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:03.740754 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-0f0caa3d-9723-48bb-8ae5-b0a2a4bedabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:03.773054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:03.804811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1198/2425] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:03.814542 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b39c8-6434-4cf6-ba1b-62087d672115 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:03.814840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b39c8-6434-4cf6-ba1b-62087d672115 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:03.815118 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b39c8-6434-4cf6-ba1b-62087d672115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:03.822675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:03.840827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-cd2b39c8-6434-4cf6-ba1b-62087d672115 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:03.842043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cd2b39c8-6434-4cf6-ba1b-62087d672115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:03.849894 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1228/2426] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:03.854160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0caa3d-9723-48bb-8ae5-b0a2a4bedabf None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:03.864963 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0caa3d-9723-48bb-8ae5-b0a2a4bedabf None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:03.870149 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998edc62-2ca8-412f-b376-8db4ada3d353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:03.870758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998edc62-2ca8-412f-b376-8db4ada3d353 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:03.871031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998edc62-2ca8-412f-b376-8db4ada3d353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:03.871296 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998edc62-2ca8-412f-b376-8db4ada3d353 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:03.871967 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998edc62-2ca8-412f-b376-8db4ada3d353 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:38:03.872267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998edc62-2ca8-412f-b376-8db4ada3d353 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:03.874777 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998edc62-2ca8-412f-b376-8db4ada3d353 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:03.889991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1199/2427] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:38:03.894309 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b39c8-6434-4cf6-ba1b-62087d672115 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:03.904455 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b39c8-6434-4cf6-ba1b-62087d672115 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:03.909130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e7d2a-2448-41a1-8853-7e0b26a7d019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:03.909726 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7d2a-2448-41a1-8853-7e0b26a7d019 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:03.910007 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7d2a-2448-41a1-8853-7e0b26a7d019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:03.910282 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7d2a-2448-41a1-8853-7e0b26a7d019 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:03.910916 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e7d2a-2448-41a1-8853-7e0b26a7d019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:03.912859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e7d2a-2448-41a1-8853-7e0b26a7d019 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:03.942714 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1229/2428] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:38:03 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:03.946753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998edc62-2ca8-412f-b376-8db4ada3d353 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:03.957372 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998edc62-2ca8-412f-b376-8db4ada3d353 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:04.016274 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e704e97-a10a-414f-8c01-e82c20a0711c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:04.016913 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e704e97-a10a-414f-8c01-e82c20a0711c admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:04.017212 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e704e97-a10a-414f-8c01-e82c20a0711c admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:04.017487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e704e97-a10a-414f-8c01-e82c20a0711c admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:04.018183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e704e97-a10a-414f-8c01-e82c20a0711c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:04.020409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e704e97-a10a-414f-8c01-e82c20a0711c admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:04.045244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1200/2429] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:38:03 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:04.046442 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e7d2a-2448-41a1-8853-7e0b26a7d019 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:04.056931 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e7d2a-2448-41a1-8853-7e0b26a7d019 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:04.062287 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff46ba-f73c-4e9d-9db2-8820f3a96f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:04.062885 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff46ba-f73c-4e9d-9db2-8820f3a96f81 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:04.063162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff46ba-f73c-4e9d-9db2-8820f3a96f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:04.063456 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff46ba-f73c-4e9d-9db2-8820f3a96f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:04.064123 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff46ba-f73c-4e9d-9db2-8820f3a96f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:04.065752 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcff46ba-f73c-4e9d-9db2-8820f3a96f81 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:04.076729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1230/2430] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:04.080851 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e704e97-a10a-414f-8c01-e82c20a0711c admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:04.093442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e704e97-a10a-414f-8c01-e82c20a0711c admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:04.099101 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968091e-2313-4a3b-aa8a-b76ceabfe149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:04.099790 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968091e-2313-4a3b-aa8a-b76ceabfe149 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:04.100130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968091e-2313-4a3b-aa8a-b76ceabfe149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:04.100473 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968091e-2313-4a3b-aa8a-b76ceabfe149 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:04.101132 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968091e-2313-4a3b-aa8a-b76ceabfe149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:04.102736 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968091e-2313-4a3b-aa8a-b76ceabfe149 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:04.114446 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1201/2431] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:04.118681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff46ba-f73c-4e9d-9db2-8820f3a96f81 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:04.129965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff46ba-f73c-4e9d-9db2-8820f3a96f81 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:04.134786 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c662f-51c1-4c68-9120-8795e9796cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:04.135380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c662f-51c1-4c68-9120-8795e9796cf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:04.135693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c662f-51c1-4c68-9120-8795e9796cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:04.135969 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3c662f-51c1-4c68-9120-8795e9796cf2 admin admin] PATH_INFO: `/v3/projects/10d8237b8556483f86ced1e56446a8f5/users/84bcadb5504a4e588cd96343ff65d4df/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:04.151061 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c662f-51c1-4c68-9120-8795e9796cf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d8237b8556483f86ced1e56446a8f5, user_id=84bcadb5504a4e588cd96343ff65d4df, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:04.154453 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3c662f-51c1-4c68-9120-8795e9796cf2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:04.168090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1231/2432] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:04 2022] PUT /identity/v3/projects/10d8237b8556483f86ced1e56446a8f5/users/84bcadb5504a4e588cd96343ff65d4df/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:04.178856 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb0beb-ebfb-40a6-bebf-929ddde84cec None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:04.179141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb0beb-ebfb-40a6-bebf-929ddde84cec None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:04.179416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acb0beb-ebfb-40a6-bebf-929ddde84cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:04.224351 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-0acb0beb-ebfb-40a6-bebf-929ddde84cec None None] MFA Rules not processed for user `84bcadb5504a4e588cd96343ff65d4df`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:04.225576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-0acb0beb-ebfb-40a6-bebf-929ddde84cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:04.298753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1202/2433] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:38:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:04.310770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c662f-51c1-4c68-9120-8795e9796cf2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:04.312530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fd3c662f-51c1-4c68-9120-8795e9796cf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:04.372415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3c662f-51c1-4c68-9120-8795e9796cf2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:04.376195 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb3474-ebb1-451a-92d2-920b72cdc457 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd8df2f3357d4660bc22366f9746f89c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:04.376814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb3474-ebb1-451a-92d2-920b72cdc457 service glance] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:04.377171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb3474-ebb1-451a-92d2-920b72cdc457 service glance] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:04.377356 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb3474-ebb1-451a-92d2-920b72cdc457 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:04.386470 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb3474-ebb1-451a-92d2-920b72cdc457 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:04.387935 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb3474-ebb1-451a-92d2-920b72cdc457 service glance] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:04.397533 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1232/2434] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:05.802617 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb0beb-ebfb-40a6-bebf-929ddde84cec None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:05.806771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-0acb0beb-ebfb-40a6-bebf-929ddde84cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:05.847443 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:05.928722 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acb0beb-ebfb-40a6-bebf-929ddde84cec None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:05.939743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19308a84-d998-4193-bc86-88171e91da21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:05.939743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19308a84-d998-4193-bc86-88171e91da21 service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:05.939743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19308a84-d998-4193-bc86-88171e91da21 service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:05.939743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19308a84-d998-4193-bc86-88171e91da21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:05.941665 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-19308a84-d998-4193-bc86-88171e91da21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:05.960117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19308a84-d998-4193-bc86-88171e91da21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:05.963993 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19308a84-d998-4193-bc86-88171e91da21 service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:05.977399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1203/2435] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:06.709275 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb3474-ebb1-451a-92d2-920b72cdc457 service glance] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:06.710866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-09eb3474-ebb1-451a-92d2-920b72cdc457 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:06.778112 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb3474-ebb1-451a-92d2-920b72cdc457 service glance] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:06.782125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df82d4f-ef37-47d7-80e5-5cbd182477b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:06.782749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df82d4f-ef37-47d7-80e5-5cbd182477b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:06.783690 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df82d4f-ef37-47d7-80e5-5cbd182477b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:06.783977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df82d4f-ef37-47d7-80e5-5cbd182477b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:06.785753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4df82d4f-ef37-47d7-80e5-5cbd182477b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:06.797805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df82d4f-ef37-47d7-80e5-5cbd182477b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:06.799977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df82d4f-ef37-47d7-80e5-5cbd182477b5 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:06.810987 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1233/2436] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:08.917247 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ea281-b515-46f8-9088-fed95d272840 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:08.917773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ea281-b515-46f8-9088-fed95d272840 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:08.917899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ea281-b515-46f8-9088-fed95d272840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:08.937211 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-f46ea281-b515-46f8-9088-fed95d272840 None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:08.938442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f46ea281-b515-46f8-9088-fed95d272840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:08.996634 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1204/2437] 10.222.0.58 () {58 vars in 1009 bytes} [Wed Aug 24 12:38:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:09.008917 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df82d4f-ef37-47d7-80e5-5cbd182477b5 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:09.010313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-4df82d4f-ef37-47d7-80e5-5cbd182477b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:09.058380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:09.084983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df82d4f-ef37-47d7-80e5-5cbd182477b5 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:09.090365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c8404-d573-4852-af79-644c9ebc3fd2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:09.091003 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c8404-d573-4852-af79-644c9ebc3fd2 service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:09.091285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c8404-d573-4852-af79-644c9ebc3fd2 service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:09.091559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c8404-d573-4852-af79-644c9ebc3fd2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:09.100242 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c8404-d573-4852-af79-644c9ebc3fd2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:09.101821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c8404-d573-4852-af79-644c9ebc3fd2 service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:09.110251 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1234/2438] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:38:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:38:18.989795 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=45 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:19.106985 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=12 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:23.350732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ea281-b515-46f8-9088-fed95d272840 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:23.365168 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ea281-b515-46f8-9088-fed95d272840 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:23.370762 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee1759b-b57e-4b3a-a646-3e9dc894803c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:23.371496 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1759b-b57e-4b3a-a646-3e9dc894803c service nova] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:23.372579 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1759b-b57e-4b3a-a646-3e9dc894803c service nova] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:23.372707 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee1759b-b57e-4b3a-a646-3e9dc894803c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:23.374470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-aee1759b-b57e-4b3a-a646-3e9dc894803c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:23.386710 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee1759b-b57e-4b3a-a646-3e9dc894803c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:23.388850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee1759b-b57e-4b3a-a646-3e9dc894803c service nova] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:23.402144 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1205/2439] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:23.553240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c8404-d573-4852-af79-644c9ebc3fd2 service swift] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:23.554671 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ca3c8404-d573-4852-af79-644c9ebc3fd2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:23.570868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c8404-d573-4852-af79-644c9ebc3fd2 service swift] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:23.577344 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb432f-938c-4f8f-be7f-9453e9c7a13a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:23.577967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb432f-938c-4f8f-be7f-9453e9c7a13a service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:23.578248 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb432f-938c-4f8f-be7f-9453e9c7a13a service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:23.578522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb432f-938c-4f8f-be7f-9453e9c7a13a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:23.580240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-15eb432f-938c-4f8f-be7f-9453e9c7a13a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:23.643832 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb432f-938c-4f8f-be7f-9453e9c7a13a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:23.645440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb432f-938c-4f8f-be7f-9453e9c7a13a service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:23.655313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1235/2440] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:25.535394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969a7a3-ce5d-45c6-a303-2fb20cba557e None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:25.536623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969a7a3-ce5d-45c6-a303-2fb20cba557e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:25.536910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969a7a3-ce5d-45c6-a303-2fb20cba557e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:25.561447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-6969a7a3-ce5d-45c6-a303-2fb20cba557e None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:25.562950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-6969a7a3-ce5d-45c6-a303-2fb20cba557e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:25.570978 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1206/2441] 10.222.0.58 () {58 vars in 967 bytes} [Wed Aug 24 12:38:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:25.585454 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb432f-938c-4f8f-be7f-9453e9c7a13a service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:25.598920 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb432f-938c-4f8f-be7f-9453e9c7a13a service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:25.604406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e6272-dfff-4ee1-b7c7-26415a622383 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:25.605230 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e6272-dfff-4ee1-b7c7-26415a622383 service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:25.605746 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e6272-dfff-4ee1-b7c7-26415a622383 service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:25.606998 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e6272-dfff-4ee1-b7c7-26415a622383 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:25.617975 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e6272-dfff-4ee1-b7c7-26415a622383 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:25.620165 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e6272-dfff-4ee1-b7c7-26415a622383 service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:25.630287 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1236/2442] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:38:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:38:34.504069 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969a7a3-ce5d-45c6-a303-2fb20cba557e None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:34.570147 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969a7a3-ce5d-45c6-a303-2fb20cba557e None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:34.575090 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18199c04-29e7-449c-a6b2-30632579a719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:34.576052 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18199c04-29e7-449c-a6b2-30632579a719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:34.576332 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18199c04-29e7-449c-a6b2-30632579a719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:34.576604 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18199c04-29e7-449c-a6b2-30632579a719 admin admin] PATH_INFO: `/v3/users/84bcadb5504a4e588cd96343ff65d4df` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:34.577721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18199c04-29e7-449c-a6b2-30632579a719 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84bcadb5504a4e588cd96343ff65d4df)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:34.579681 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18199c04-29e7-449c-a6b2-30632579a719 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:34.627086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-18199c04-29e7-449c-a6b2-30632579a719 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84bcadb5504a4e588cd96343ff65d4df'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:34.627776 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-18199c04-29e7-449c-a6b2-30632579a719 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84bcadb5504a4e588cd96343ff65d4df'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:34.628389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-18199c04-29e7-449c-a6b2-30632579a719 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84bcadb5504a4e588cd96343ff65d4df'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:34.635283 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-18199c04-29e7-449c-a6b2-30632579a719 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84bcadb5504a4e588cd96343ff65d4df'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:34.642088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-18199c04-29e7-449c-a6b2-30632579a719 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84bcadb5504a4e588cd96343ff65d4df'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:34.659980 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1207/2443] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:38:34 2022] DELETE /identity/v3/users/84bcadb5504a4e588cd96343ff65d4df => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:34.664663 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11549e0c-bcba-42ca-b88d-9a0c78b2cb6f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:34.665123 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11549e0c-bcba-42ca-b88d-9a0c78b2cb6f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:34.665343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11549e0c-bcba-42ca-b88d-9a0c78b2cb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:34.685750 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-11549e0c-bcba-42ca-b88d-9a0c78b2cb6f None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:34.686954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-11549e0c-bcba-42ca-b88d-9a0c78b2cb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:34.745971 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1237/2444] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:34.755822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18199c04-29e7-449c-a6b2-30632579a719 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:34.757268 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-18199c04-29e7-449c-a6b2-30632579a719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:34.814427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18199c04-29e7-449c-a6b2-30632579a719 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:34.818052 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfb455-22e1-477d-b5a3-de424d9a897f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:34.818650 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfb455-22e1-477d-b5a3-de424d9a897f service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:34.818932 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfb455-22e1-477d-b5a3-de424d9a897f service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:34.819205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfb455-22e1-477d-b5a3-de424d9a897f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:34.828929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfb455-22e1-477d-b5a3-de424d9a897f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:34.830348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfb455-22e1-477d-b5a3-de424d9a897f service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:34.839733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1208/2445] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:35.109184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11549e0c-bcba-42ca-b88d-9a0c78b2cb6f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:35.110605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-11549e0c-bcba-42ca-b88d-9a0c78b2cb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:35.124324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11549e0c-bcba-42ca-b88d-9a0c78b2cb6f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:35.128897 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54790883-71bf-4241-a1ea-b963ce55e75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:35.129533 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54790883-71bf-4241-a1ea-b963ce55e75c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.129814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54790883-71bf-4241-a1ea-b963ce55e75c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.130092 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54790883-71bf-4241-a1ea-b963ce55e75c admin admin] PATH_INFO: `/v3/projects/10d8237b8556483f86ced1e56446a8f5` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.131192 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54790883-71bf-4241-a1ea-b963ce55e75c admin admin] RBAC: Authorizing `identity:delete_project(project_id=10d8237b8556483f86ced1e56446a8f5)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:35.133130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54790883-71bf-4241-a1ea-b963ce55e75c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:35.179840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-54790883-71bf-4241-a1ea-b963ce55e75c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10d8237b8556483f86ced1e56446a8f5', 'request_id': 'req-54790883-71bf-4241-a1ea-b963ce55e75c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:35.185914 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-54790883-71bf-4241-a1ea-b963ce55e75c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10d8237b8556483f86ced1e56446a8f5', 'request_id': 'req-54790883-71bf-4241-a1ea-b963ce55e75c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:35.193329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-54790883-71bf-4241-a1ea-b963ce55e75c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10d8237b8556483f86ced1e56446a8f5', 'request_id': 'req-54790883-71bf-4241-a1ea-b963ce55e75c'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:35.205803 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-54790883-71bf-4241-a1ea-b963ce55e75c admin admin] The token cache is being invalidate because project 10d8237b8556483f86ced1e56446a8f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:38:35.206094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-54790883-71bf-4241-a1ea-b963ce55e75c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:35.206724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-54790883-71bf-4241-a1ea-b963ce55e75c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:38:35.208266 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1238/2446] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:38:35 2022] DELETE /identity/v3/projects/10d8237b8556483f86ced1e56446a8f5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:35.213068 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121b24f-d2e2-4733-85ab-4a4093e7bcca None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.213353 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121b24f-d2e2-4733-85ab-4a4093e7bcca None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.213631 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a121b24f-d2e2-4733-85ab-4a4093e7bcca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.221145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:35.235270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-a121b24f-d2e2-4733-85ab-4a4093e7bcca None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:35.236521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a121b24f-d2e2-4733-85ab-4a4093e7bcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:35.298493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1209/2447] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:35.306466 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451800d-e8de-427b-85d0-9cecc722cff1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.306750 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451800d-e8de-427b-85d0-9cecc722cff1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.307032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451800d-e8de-427b-85d0-9cecc722cff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.328271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9451800d-e8de-427b-85d0-9cecc722cff1 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:35.329496 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9451800d-e8de-427b-85d0-9cecc722cff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:35.337583 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1239/2448] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:35.341901 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121b24f-d2e2-4733-85ab-4a4093e7bcca None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:35.351351 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a121b24f-d2e2-4733-85ab-4a4093e7bcca None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:35.355969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f1275-5226-4aae-9a06-8c5c4479bb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:35.356558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f1275-5226-4aae-9a06-8c5c4479bb1d admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.356866 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f1275-5226-4aae-9a06-8c5c4479bb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.357139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f1275-5226-4aae-9a06-8c5c4479bb1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.357795 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f1275-5226-4aae-9a06-8c5c4479bb1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:38:35.358081 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f1275-5226-4aae-9a06-8c5c4479bb1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:35.359679 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f1275-5226-4aae-9a06-8c5c4479bb1d admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:35.370963 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1210/2449] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:38:35.376144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451800d-e8de-427b-85d0-9cecc722cff1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:35.380942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:35.388225 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451800d-e8de-427b-85d0-9cecc722cff1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:35.394263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7200321-2e09-418b-abf7-4cdf0ebfacb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:35.394919 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7200321-2e09-418b-abf7-4cdf0ebfacb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.395201 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7200321-2e09-418b-abf7-4cdf0ebfacb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.395691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7200321-2e09-418b-abf7-4cdf0ebfacb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.396557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7200321-2e09-418b-abf7-4cdf0ebfacb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:35.398669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7200321-2e09-418b-abf7-4cdf0ebfacb6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:35.426033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1240/2450] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:38:35 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:38:35.430383 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f1275-5226-4aae-9a06-8c5c4479bb1d admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:35.440743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f1275-5226-4aae-9a06-8c5c4479bb1d admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:35.494661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615cecf5-9375-465e-b4b9-3227daa9dc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:35.495254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cecf5-9375-465e-b4b9-3227daa9dc5d admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.495528 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cecf5-9375-465e-b4b9-3227daa9dc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.495824 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cecf5-9375-465e-b4b9-3227daa9dc5d admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.497075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615cecf5-9375-465e-b4b9-3227daa9dc5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:35.498883 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615cecf5-9375-465e-b4b9-3227daa9dc5d admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:35.520887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1211/2451] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:38:35 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:38:35.526359 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7200321-2e09-418b-abf7-4cdf0ebfacb6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:35.537032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7200321-2e09-418b-abf7-4cdf0ebfacb6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:35.541540 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de4740-c1b4-464e-9da4-4300517c651b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:35.542132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de4740-c1b4-464e-9da4-4300517c651b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.542409 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de4740-c1b4-464e-9da4-4300517c651b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.542681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de4740-c1b4-464e-9da4-4300517c651b admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.543677 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de4740-c1b4-464e-9da4-4300517c651b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:35.545307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de4740-c1b4-464e-9da4-4300517c651b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:35.555050 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1241/2452] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:35.559531 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615cecf5-9375-465e-b4b9-3227daa9dc5d admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:35.569350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615cecf5-9375-465e-b4b9-3227daa9dc5d admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:35.574032 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1fb0f-8d17-4443-9539-107ed02c9c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:35.575720 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1fb0f-8d17-4443-9539-107ed02c9c19 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.576009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1fb0f-8d17-4443-9539-107ed02c9c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.576286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1fb0f-8d17-4443-9539-107ed02c9c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.576928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1fb0f-8d17-4443-9539-107ed02c9c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:35.578519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1fb0f-8d17-4443-9539-107ed02c9c19 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:35.589040 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1212/2453] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:38:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:38:35.593176 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de4740-c1b4-464e-9da4-4300517c651b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:35.602930 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de4740-c1b4-464e-9da4-4300517c651b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:35.607603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63ed27-246e-4786-a40f-cb5ba4207cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:35.608269 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63ed27-246e-4786-a40f-cb5ba4207cae admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.608549 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63ed27-246e-4786-a40f-cb5ba4207cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.608830 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63ed27-246e-4786-a40f-cb5ba4207cae admin admin] PATH_INFO: `/v3/projects/474008375491499b85702c8824bd1981/users/9c40bc0027e94421a1da7b8016307b67/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.624728 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63ed27-246e-4786-a40f-cb5ba4207cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=474008375491499b85702c8824bd1981, user_id=9c40bc0027e94421a1da7b8016307b67, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:35.627365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63ed27-246e-4786-a40f-cb5ba4207cae admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:35.640369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1242/2454] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:38:35 2022] PUT /identity/v3/projects/474008375491499b85702c8824bd1981/users/9c40bc0027e94421a1da7b8016307b67/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:38:35.645386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d027596-d906-41dc-abe4-31448ab09f00 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.645677 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d027596-d906-41dc-abe4-31448ab09f00 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.645953 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d027596-d906-41dc-abe4-31448ab09f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.664242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-0d027596-d906-41dc-abe4-31448ab09f00 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:35.665437 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-0d027596-d906-41dc-abe4-31448ab09f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:35.724472 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1213/2455] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:38:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:38:35.734769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63ed27-246e-4786-a40f-cb5ba4207cae admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:35.736597 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ee63ed27-246e-4786-a40f-cb5ba4207cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:35.795009 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63ed27-246e-4786-a40f-cb5ba4207cae admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:35.798609 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9b811-bf05-46c6-824c-773ec579dbf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:35.799213 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9b811-bf05-46c6-824c-773ec579dbf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:35.799498 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9b811-bf05-46c6-824c-773ec579dbf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:35.799803 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9b811-bf05-46c6-824c-773ec579dbf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:35.809396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9b811-bf05-46c6-824c-773ec579dbf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:35.810989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9b811-bf05-46c6-824c-773ec579dbf6 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:35.821171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1243/2456] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:36.485634 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a3e99-6e3b-481a-846c-a2de0909eb0b None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:36.485924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a3e99-6e3b-481a-846c-a2de0909eb0b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:36.486205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a3e99-6e3b-481a-846c-a2de0909eb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:36.506704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:36.518540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-512a3e99-6e3b-481a-846c-a2de0909eb0b None None] MFA Rules not processed for user `9c40bc0027e94421a1da7b8016307b67`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:36.519893 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-512a3e99-6e3b-481a-846c-a2de0909eb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:36.589323 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1214/2457] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:38:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:36.600677 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9b811-bf05-46c6-824c-773ec579dbf6 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:36.602114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-aed9b811-bf05-46c6-824c-773ec579dbf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:36.663022 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9b811-bf05-46c6-824c-773ec579dbf6 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:36.667216 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8603a7e-d15d-48ad-82bb-d46a85f4da67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:36.668177 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8603a7e-d15d-48ad-82bb-d46a85f4da67 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:36.668487 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8603a7e-d15d-48ad-82bb-d46a85f4da67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:36.668770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8603a7e-d15d-48ad-82bb-d46a85f4da67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:36.679083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8603a7e-d15d-48ad-82bb-d46a85f4da67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:36.680601 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8603a7e-d15d-48ad-82bb-d46a85f4da67 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:36.690090 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1244/2458] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:38.044318 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddc733-86e4-4c2c-99ad-9a8dddee07ee None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:38.044851 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddc733-86e4-4c2c-99ad-9a8dddee07ee None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:38.044851 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cddc733-86e4-4c2c-99ad-9a8dddee07ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:38.065863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-9cddc733-86e4-4c2c-99ad-9a8dddee07ee None None] MFA Rules not processed for user `9c40bc0027e94421a1da7b8016307b67`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:38:38.067084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-9cddc733-86e4-4c2c-99ad-9a8dddee07ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:38.073944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1215/2459] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:38:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:38:41.910956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8603a7e-d15d-48ad-82bb-d46a85f4da67 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:41.912840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e8603a7e-d15d-48ad-82bb-d46a85f4da67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:41.979314 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:42.007496 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8603a7e-d15d-48ad-82bb-d46a85f4da67 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:42.013934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdafabe-163b-473e-8933-30f6bef84e09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:42.014551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdafabe-163b-473e-8933-30f6bef84e09 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:42.014832 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdafabe-163b-473e-8933-30f6bef84e09 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:42.015107 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdafabe-163b-473e-8933-30f6bef84e09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:42.016893 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-efdafabe-163b-473e-8933-30f6bef84e09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:42.033011 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdafabe-163b-473e-8933-30f6bef84e09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:42.034596 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdafabe-163b-473e-8933-30f6bef84e09 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:42.046762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1245/2460] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:48.058711 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=31 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:52.041951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=11 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:38:56.642494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddc733-86e4-4c2c-99ad-9a8dddee07ee None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:56.652128 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cddc733-86e4-4c2c-99ad-9a8dddee07ee None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:38:56.656120 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f15c9-73ca-46aa-88d8-843a87bddf20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:38:56.656715 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f15c9-73ca-46aa-88d8-843a87bddf20 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:38:56.656996 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f15c9-73ca-46aa-88d8-843a87bddf20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:38:56.657267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f15c9-73ca-46aa-88d8-843a87bddf20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:38:56.658972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-211f15c9-73ca-46aa-88d8-843a87bddf20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:38:56.670607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f15c9-73ca-46aa-88d8-843a87bddf20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:38:56.672061 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f15c9-73ca-46aa-88d8-843a87bddf20 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:38:56.681558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1216/2461] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:38:59.938866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdafabe-163b-473e-8933-30f6bef84e09 service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:38:59.940694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-efdafabe-163b-473e-8933-30f6bef84e09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:00.012843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f15c9-73ca-46aa-88d8-843a87bddf20 service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:00.014978 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-211f15c9-73ca-46aa-88d8-843a87bddf20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:00.032310 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f15c9-73ca-46aa-88d8-843a87bddf20 service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:00.039966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3d3ee-8c93-4a08-8a4d-069ba4615b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:00.042054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3d3ee-8c93-4a08-8a4d-069ba4615b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:00.042481 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3d3ee-8c93-4a08-8a4d-069ba4615b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:00.042960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3d3ee-8c93-4a08-8a4d-069ba4615b24 admin admin] PATH_INFO: `/v3/users/9aa753d1e2c4430498d5b28f0304952f` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:00.045017 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3d3ee-8c93-4a08-8a4d-069ba4615b24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa753d1e2c4430498d5b28f0304952f)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:00.047673 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3d3ee-8c93-4a08-8a4d-069ba4615b24 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:00.051730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdafabe-163b-473e-8933-30f6bef84e09 service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:00.063524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8eb642-0f4e-4dab-9e12-323fae608048 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ba45e3cff74962965b248f9f017f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:00.064551 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8eb642-0f4e-4dab-9e12-323fae608048 service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:00.065213 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8eb642-0f4e-4dab-9e12-323fae608048 service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:00.065638 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8eb642-0f4e-4dab-9e12-323fae608048 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:00.067952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3e8eb642-0f4e-4dab-9e12-323fae608048 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:00.082614 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8eb642-0f4e-4dab-9e12-323fae608048 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:00.084535 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8eb642-0f4e-4dab-9e12-323fae608048 service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:00.095535 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1246/2462] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:38:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:00.127583 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3ae3d3ee-8c93-4a08-8a4d-069ba4615b24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa753d1e2c4430498d5b28f0304952f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:00.128412 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3ae3d3ee-8c93-4a08-8a4d-069ba4615b24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa753d1e2c4430498d5b28f0304952f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:00.128993 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3ae3d3ee-8c93-4a08-8a4d-069ba4615b24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa753d1e2c4430498d5b28f0304952f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:00.143180 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3ae3d3ee-8c93-4a08-8a4d-069ba4615b24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa753d1e2c4430498d5b28f0304952f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:00.152841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3ae3d3ee-8c93-4a08-8a4d-069ba4615b24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa753d1e2c4430498d5b28f0304952f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:00.177315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1217/2463] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:00 2022] DELETE /identity/v3/users/9aa753d1e2c4430498d5b28f0304952f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:00.367688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8eb642-0f4e-4dab-9e12-323fae608048 service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:00.379124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8eb642-0f4e-4dab-9e12-323fae608048 service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:00.448347 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e02ef4-2938-4a35-852e-6a99849b191f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:00.449139 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e02ef4-2938-4a35-852e-6a99849b191f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:00.449672 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e02ef4-2938-4a35-852e-6a99849b191f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:00.449914 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e02ef4-2938-4a35-852e-6a99849b191f admin admin] PATH_INFO: `/v3/projects/f3a2fec602684ab1a454b8da589b4787` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:00.451486 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e02ef4-2938-4a35-852e-6a99849b191f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3a2fec602684ab1a454b8da589b4787)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:00.453476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e02ef4-2938-4a35-852e-6a99849b191f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:00.502825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-81e02ef4-2938-4a35-852e-6a99849b191f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3a2fec602684ab1a454b8da589b4787', 'request_id': 'req-81e02ef4-2938-4a35-852e-6a99849b191f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:00.509183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-81e02ef4-2938-4a35-852e-6a99849b191f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3a2fec602684ab1a454b8da589b4787', 'request_id': 'req-81e02ef4-2938-4a35-852e-6a99849b191f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:00.517337 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-81e02ef4-2938-4a35-852e-6a99849b191f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3a2fec602684ab1a454b8da589b4787', 'request_id': 'req-81e02ef4-2938-4a35-852e-6a99849b191f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:00.529951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-81e02ef4-2938-4a35-852e-6a99849b191f admin admin] The token cache is being invalidate because project f3a2fec602684ab1a454b8da589b4787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:00.530249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-81e02ef4-2938-4a35-852e-6a99849b191f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:00.530903 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-81e02ef4-2938-4a35-852e-6a99849b191f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:00.532539 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1247/2464] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:00 2022] DELETE /identity/v3/projects/f3a2fec602684ab1a454b8da589b4787 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:00.537833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9788a-fed2-4b94-a38a-003799bdfaa2 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:00.538360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9788a-fed2-4b94-a38a-003799bdfaa2 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:00.538822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9788a-fed2-4b94-a38a-003799bdfaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:00.559810 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-97d9788a-fed2-4b94-a38a-003799bdfaa2 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:00.561054 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-97d9788a-fed2-4b94-a38a-003799bdfaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:00.627321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1218/2465] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:00.635953 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f442fef-cb32-4d66-8f40-da833037a4a4 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:00.636243 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f442fef-cb32-4d66-8f40-da833037a4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:00.636522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f442fef-cb32-4d66-8f40-da833037a4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:00.659697 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8f442fef-cb32-4d66-8f40-da833037a4a4 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:00.660944 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8f442fef-cb32-4d66-8f40-da833037a4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:00.672991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1248/2466] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:00.674187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9788a-fed2-4b94-a38a-003799bdfaa2 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:00.683895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9788a-fed2-4b94-a38a-003799bdfaa2 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:00.688655 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456cb860-004c-4e32-9138-3e1f944da8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:00.689261 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456cb860-004c-4e32-9138-3e1f944da8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:00.689543 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456cb860-004c-4e32-9138-3e1f944da8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:00.689821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456cb860-004c-4e32-9138-3e1f944da8a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:00.690464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456cb860-004c-4e32-9138-3e1f944da8a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:39:00.690767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456cb860-004c-4e32-9138-3e1f944da8a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:00.692394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456cb860-004c-4e32-9138-3e1f944da8a4 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:00.705115 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1219/2467] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:39:00.709340 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f442fef-cb32-4d66-8f40-da833037a4a4 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:00.723311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f442fef-cb32-4d66-8f40-da833037a4a4 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:00.728157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f86f5c-c06d-4c8c-9a4c-6e724981892d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:00.728761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f86f5c-c06d-4c8c-9a4c-6e724981892d admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:00.729083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f86f5c-c06d-4c8c-9a4c-6e724981892d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:00.729384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f86f5c-c06d-4c8c-9a4c-6e724981892d admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:00.730235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f86f5c-c06d-4c8c-9a4c-6e724981892d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:00.732410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f86f5c-c06d-4c8c-9a4c-6e724981892d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:00.744477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:00.763732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1249/2468] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:00 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:00.766916 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456cb860-004c-4e32-9138-3e1f944da8a4 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:00.777219 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456cb860-004c-4e32-9138-3e1f944da8a4 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:00.826402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:00.846020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a4fe5-93d5-49fb-93b9-f0aa3f0b0865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:00.846908 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a4fe5-93d5-49fb-93b9-f0aa3f0b0865 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:00.847186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a4fe5-93d5-49fb-93b9-f0aa3f0b0865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:00.847457 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a4fe5-93d5-49fb-93b9-f0aa3f0b0865 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:00.848115 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a4fe5-93d5-49fb-93b9-f0aa3f0b0865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:00.850230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a4fe5-93d5-49fb-93b9-f0aa3f0b0865 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:00.885990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1220/2469] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:00 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:00.890180 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f86f5c-c06d-4c8c-9a4c-6e724981892d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:00.906468 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f86f5c-c06d-4c8c-9a4c-6e724981892d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:00.911937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe240d8-4286-45d7-834c-d4fc72e76874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:00.912867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe240d8-4286-45d7-834c-d4fc72e76874 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:00.913430 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe240d8-4286-45d7-834c-d4fc72e76874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:00.913759 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe240d8-4286-45d7-834c-d4fc72e76874 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:00.914712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe240d8-4286-45d7-834c-d4fc72e76874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:00.918323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe240d8-4286-45d7-834c-d4fc72e76874 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:00.932215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1250/2470] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:00.933575 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a4fe5-93d5-49fb-93b9-f0aa3f0b0865 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:00.944262 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a4fe5-93d5-49fb-93b9-f0aa3f0b0865 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:00.951579 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355e104-7278-467e-9671-913250b3a0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:00.952220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355e104-7278-467e-9671-913250b3a0be admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:00.952507 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355e104-7278-467e-9671-913250b3a0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:00.952785 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355e104-7278-467e-9671-913250b3a0be admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:00.953437 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355e104-7278-467e-9671-913250b3a0be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:00.955051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355e104-7278-467e-9671-913250b3a0be admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:00.969301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1221/2471] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:00.973637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe240d8-4286-45d7-834c-d4fc72e76874 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:00.984600 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe240d8-4286-45d7-834c-d4fc72e76874 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:00.990872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c12a2-58c0-4305-9f28-aad963613e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:00.991893 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c12a2-58c0-4305-9f28-aad963613e9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:00.992667 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c12a2-58c0-4305-9f28-aad963613e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:00.993135 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c12a2-58c0-4305-9f28-aad963613e9b admin admin] PATH_INFO: `/v3/projects/3e2f7c6ccdf048d8b4db223de2ded668/users/b617a6fcd083498f81373bff6673c094/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:01.011526 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c12a2-58c0-4305-9f28-aad963613e9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e2f7c6ccdf048d8b4db223de2ded668, user_id=b617a6fcd083498f81373bff6673c094, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:01.014432 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0c12a2-58c0-4305-9f28-aad963613e9b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:01.030459 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1251/2472] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:00 2022] PUT /identity/v3/projects/3e2f7c6ccdf048d8b4db223de2ded668/users/b617a6fcd083498f81373bff6673c094/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:01.040914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bacb1d-ba79-42ef-8f93-86fb60e28710 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:01.041540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bacb1d-ba79-42ef-8f93-86fb60e28710 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:01.041540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bacb1d-ba79-42ef-8f93-86fb60e28710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:01.070843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-23bacb1d-ba79-42ef-8f93-86fb60e28710 None None] MFA Rules not processed for user `b617a6fcd083498f81373bff6673c094`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:01.072222 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-23bacb1d-ba79-42ef-8f93-86fb60e28710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:01.139217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1222/2473] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:01.149816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c12a2-58c0-4305-9f28-aad963613e9b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:01.151507 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3f0c12a2-58c0-4305-9f28-aad963613e9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:01.224997 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0c12a2-58c0-4305-9f28-aad963613e9b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:01.230446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fe5a0-473c-4798-b5f1-f209948eff43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:01.231037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fe5a0-473c-4798-b5f1-f209948eff43 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:01.231312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fe5a0-473c-4798-b5f1-f209948eff43 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:01.231581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fe5a0-473c-4798-b5f1-f209948eff43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:01.239895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fe5a0-473c-4798-b5f1-f209948eff43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:01.241471 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fe5a0-473c-4798-b5f1-f209948eff43 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:01.250438 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1252/2474] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:09.616925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bacb1d-ba79-42ef-8f93-86fb60e28710 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:09.686407 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bacb1d-ba79-42ef-8f93-86fb60e28710 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:09.691259 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcf635-d70d-4309-946f-56e73fe7e205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:09.692046 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcf635-d70d-4309-946f-56e73fe7e205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:09.692448 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcf635-d70d-4309-946f-56e73fe7e205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:09.692839 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdcf635-d70d-4309-946f-56e73fe7e205 admin admin] PATH_INFO: `/v3/users/b617a6fcd083498f81373bff6673c094` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:09.694092 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdcf635-d70d-4309-946f-56e73fe7e205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b617a6fcd083498f81373bff6673c094)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:09.696179 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdcf635-d70d-4309-946f-56e73fe7e205 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:09.761933 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-dbdcf635-d70d-4309-946f-56e73fe7e205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b617a6fcd083498f81373bff6673c094'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:09.762648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-dbdcf635-d70d-4309-946f-56e73fe7e205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b617a6fcd083498f81373bff6673c094'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:09.763268 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-dbdcf635-d70d-4309-946f-56e73fe7e205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b617a6fcd083498f81373bff6673c094'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:09.770767 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-dbdcf635-d70d-4309-946f-56e73fe7e205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b617a6fcd083498f81373bff6673c094'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:09.773131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:09.781459 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-dbdcf635-d70d-4309-946f-56e73fe7e205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b617a6fcd083498f81373bff6673c094'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:09.809205 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1223/2475] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:09 2022] DELETE /identity/v3/users/b617a6fcd083498f81373bff6673c094 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:09.814986 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef16622-5082-424d-aa7f-c5f91f2dfe83 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:09.815540 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef16622-5082-424d-aa7f-c5f91f2dfe83 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:09.815741 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef16622-5082-424d-aa7f-c5f91f2dfe83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:09.840885 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1ef16622-5082-424d-aa7f-c5f91f2dfe83 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:09.843435 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1ef16622-5082-424d-aa7f-c5f91f2dfe83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:09.947279 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1253/2476] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:09.960051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcf635-d70d-4309-946f-56e73fe7e205 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:09.963903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-dbdcf635-d70d-4309-946f-56e73fe7e205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:10.048374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdcf635-d70d-4309-946f-56e73fe7e205 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:10.053373 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae7ce1-2161-4518-b6b4-3e1851939bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:10.054070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae7ce1-2161-4518-b6b4-3e1851939bae service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:10.054362 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae7ce1-2161-4518-b6b4-3e1851939bae service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:10.054641 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae7ce1-2161-4518-b6b4-3e1851939bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:10.066017 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae7ce1-2161-4518-b6b4-3e1851939bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:10.067531 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae7ce1-2161-4518-b6b4-3e1851939bae service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:10.083875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1224/2477] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:10.467597 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef16622-5082-424d-aa7f-c5f91f2dfe83 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:10.469903 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1ef16622-5082-424d-aa7f-c5f91f2dfe83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:10.484265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef16622-5082-424d-aa7f-c5f91f2dfe83 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:10.489334 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd8e30-a8fa-4c62-a011-c56b6b884a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:10.489991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd8e30-a8fa-4c62-a011-c56b6b884a0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:10.490290 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd8e30-a8fa-4c62-a011-c56b6b884a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:10.490552 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd8e30-a8fa-4c62-a011-c56b6b884a0d admin admin] PATH_INFO: `/v3/projects/3e2f7c6ccdf048d8b4db223de2ded668` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:10.491742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd8e30-a8fa-4c62-a011-c56b6b884a0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e2f7c6ccdf048d8b4db223de2ded668)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:10.493679 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd8e30-a8fa-4c62-a011-c56b6b884a0d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:10.550616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:10.563843 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4dfd8e30-a8fa-4c62-a011-c56b6b884a0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e2f7c6ccdf048d8b4db223de2ded668', 'request_id': 'req-4dfd8e30-a8fa-4c62-a011-c56b6b884a0d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:10.572242 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4dfd8e30-a8fa-4c62-a011-c56b6b884a0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e2f7c6ccdf048d8b4db223de2ded668', 'request_id': 'req-4dfd8e30-a8fa-4c62-a011-c56b6b884a0d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:10.584699 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4dfd8e30-a8fa-4c62-a011-c56b6b884a0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e2f7c6ccdf048d8b4db223de2ded668', 'request_id': 'req-4dfd8e30-a8fa-4c62-a011-c56b6b884a0d'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:10.601693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4dfd8e30-a8fa-4c62-a011-c56b6b884a0d admin admin] The token cache is being invalidate because project 3e2f7c6ccdf048d8b4db223de2ded668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:10.601990 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4dfd8e30-a8fa-4c62-a011-c56b6b884a0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:10.602735 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-4dfd8e30-a8fa-4c62-a011-c56b6b884a0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:10.608135 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1254/2478] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:10 2022] DELETE /identity/v3/projects/3e2f7c6ccdf048d8b4db223de2ded668 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:10.609828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794fe9c-425a-4a7a-b03b-ac743a6b91cf None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:10.610115 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794fe9c-425a-4a7a-b03b-ac743a6b91cf None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:10.610390 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1794fe9c-425a-4a7a-b03b-ac743a6b91cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:10.637946 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-1794fe9c-425a-4a7a-b03b-ac743a6b91cf None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:10.639430 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1794fe9c-425a-4a7a-b03b-ac743a6b91cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:10.710608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1225/2479] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:10.720098 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac678f1-7615-4e70-8feb-3494f6f011f1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:10.720439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac678f1-7615-4e70-8feb-3494f6f011f1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:10.720719 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac678f1-7615-4e70-8feb-3494f6f011f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:10.742627 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1ac678f1-7615-4e70-8feb-3494f6f011f1 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:10.743859 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1ac678f1-7615-4e70-8feb-3494f6f011f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:10.752773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1255/2480] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:10.757024 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794fe9c-425a-4a7a-b03b-ac743a6b91cf None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:10.767545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1794fe9c-425a-4a7a-b03b-ac743a6b91cf None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:10.772395 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66a5fe-b74c-4500-b5ff-13064cb923bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:10.773005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66a5fe-b74c-4500-b5ff-13064cb923bb admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:10.773285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66a5fe-b74c-4500-b5ff-13064cb923bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:10.773564 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66a5fe-b74c-4500-b5ff-13064cb923bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:10.774251 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66a5fe-b74c-4500-b5ff-13064cb923bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:39:10.774552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66a5fe-b74c-4500-b5ff-13064cb923bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:10.776328 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66a5fe-b74c-4500-b5ff-13064cb923bb admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:10.798395 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1226/2481] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:39:10.802624 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac678f1-7615-4e70-8feb-3494f6f011f1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:10.813368 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac678f1-7615-4e70-8feb-3494f6f011f1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:10.818292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6cc20-9e90-4537-9835-da6ca12d0f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:10.818881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6cc20-9e90-4537-9835-da6ca12d0f4c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:10.819168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6cc20-9e90-4537-9835-da6ca12d0f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:10.819441 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6cc20-9e90-4537-9835-da6ca12d0f4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:10.820109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6cc20-9e90-4537-9835-da6ca12d0f4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:10.822006 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6cc20-9e90-4537-9835-da6ca12d0f4c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:10.852401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1256/2482] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:10 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:10.855454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66a5fe-b74c-4500-b5ff-13064cb923bb admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:10.866507 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66a5fe-b74c-4500-b5ff-13064cb923bb admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:10.927515 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ffd8e1-88f2-4126-9320-c27bd874b118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:10.928173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffd8e1-88f2-4126-9320-c27bd874b118 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:10.928453 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffd8e1-88f2-4126-9320-c27bd874b118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:10.928726 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffd8e1-88f2-4126-9320-c27bd874b118 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:10.929367 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ffd8e1-88f2-4126-9320-c27bd874b118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:10.931299 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ffd8e1-88f2-4126-9320-c27bd874b118 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:10.942850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:10.961704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1227/2483] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:10 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:10.962896 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6cc20-9e90-4537-9835-da6ca12d0f4c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:10.974217 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6cc20-9e90-4537-9835-da6ca12d0f4c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:10.979350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9502888-ea8e-4814-8f0f-cc2d96736c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:10.980094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9502888-ea8e-4814-8f0f-cc2d96736c64 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:10.980484 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9502888-ea8e-4814-8f0f-cc2d96736c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:10.980853 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9502888-ea8e-4814-8f0f-cc2d96736c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:10.981816 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9502888-ea8e-4814-8f0f-cc2d96736c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:10.983986 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9502888-ea8e-4814-8f0f-cc2d96736c64 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:10.998243 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1257/2484] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:11.002371 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ffd8e1-88f2-4126-9320-c27bd874b118 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:11.016757 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ffd8e1-88f2-4126-9320-c27bd874b118 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:11.021635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e8b63-3586-458b-a2cf-41bd3c9304fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:11.022391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e8b63-3586-458b-a2cf-41bd3c9304fb admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:11.022676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e8b63-3586-458b-a2cf-41bd3c9304fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:11.022951 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e8b63-3586-458b-a2cf-41bd3c9304fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:11.023778 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e8b63-3586-458b-a2cf-41bd3c9304fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:11.025443 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e8b63-3586-458b-a2cf-41bd3c9304fb admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:11.037483 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1228/2485] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:39:11.041873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9502888-ea8e-4814-8f0f-cc2d96736c64 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:11.053269 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9502888-ea8e-4814-8f0f-cc2d96736c64 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:11.058022 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f367122-7384-4d81-a1cc-085d8ac16647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:11.058702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f367122-7384-4d81-a1cc-085d8ac16647 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:11.058984 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f367122-7384-4d81-a1cc-085d8ac16647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:11.059262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f367122-7384-4d81-a1cc-085d8ac16647 admin admin] PATH_INFO: `/v3/projects/179f6e9bbf1d4f6bb7f57f88d5941ac7/users/69a436b39b324fcc81b72b6ce7d7190c/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:11.077895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f367122-7384-4d81-a1cc-085d8ac16647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=179f6e9bbf1d4f6bb7f57f88d5941ac7, user_id=69a436b39b324fcc81b72b6ce7d7190c, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:11.080635 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f367122-7384-4d81-a1cc-085d8ac16647 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:11.094546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1258/2486] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:11 2022] PUT /identity/v3/projects/179f6e9bbf1d4f6bb7f57f88d5941ac7/users/69a436b39b324fcc81b72b6ce7d7190c/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:11.104821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257fe16-197b-449e-8809-902d52b68cee None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:11.105176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257fe16-197b-449e-8809-902d52b68cee None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:11.105502 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257fe16-197b-449e-8809-902d52b68cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:11.135331 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-3257fe16-197b-449e-8809-902d52b68cee None None] MFA Rules not processed for user `69a436b39b324fcc81b72b6ce7d7190c`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:11.136601 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-3257fe16-197b-449e-8809-902d52b68cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:11.233995 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1229/2487] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:11.246232 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f367122-7384-4d81-a1cc-085d8ac16647 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:11.249337 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-5f367122-7384-4d81-a1cc-085d8ac16647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:11.342909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f367122-7384-4d81-a1cc-085d8ac16647 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:11.348982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44484e76-7fae-482d-bfb3-02faddaf472d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:11.349590 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44484e76-7fae-482d-bfb3-02faddaf472d service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:11.349876 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44484e76-7fae-482d-bfb3-02faddaf472d service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:11.350151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44484e76-7fae-482d-bfb3-02faddaf472d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:11.360132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44484e76-7fae-482d-bfb3-02faddaf472d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:11.361726 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44484e76-7fae-482d-bfb3-02faddaf472d service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:11.372863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1259/2488] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:11.596212 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257fe16-197b-449e-8809-902d52b68cee None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:11.659490 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257fe16-197b-449e-8809-902d52b68cee None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:11.664438 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afd5ed-caad-4263-bf02-a7c6eb6a66eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:11.665069 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afd5ed-caad-4263-bf02-a7c6eb6a66eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:11.665608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afd5ed-caad-4263-bf02-a7c6eb6a66eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:11.665885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74afd5ed-caad-4263-bf02-a7c6eb6a66eb admin admin] PATH_INFO: `/v3/users/69a436b39b324fcc81b72b6ce7d7190c` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:11.666991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74afd5ed-caad-4263-bf02-a7c6eb6a66eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=69a436b39b324fcc81b72b6ce7d7190c)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:11.669955 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74afd5ed-caad-4263-bf02-a7c6eb6a66eb admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:11.715115 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-74afd5ed-caad-4263-bf02-a7c6eb6a66eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69a436b39b324fcc81b72b6ce7d7190c'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:11.716323 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-74afd5ed-caad-4263-bf02-a7c6eb6a66eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69a436b39b324fcc81b72b6ce7d7190c'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:11.717040 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-74afd5ed-caad-4263-bf02-a7c6eb6a66eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69a436b39b324fcc81b72b6ce7d7190c'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:11.723980 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-74afd5ed-caad-4263-bf02-a7c6eb6a66eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69a436b39b324fcc81b72b6ce7d7190c'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:11.731062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-74afd5ed-caad-4263-bf02-a7c6eb6a66eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69a436b39b324fcc81b72b6ce7d7190c'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:11.739726 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:11.750903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1230/2489] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:11 2022] DELETE /identity/v3/users/69a436b39b324fcc81b72b6ce7d7190c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:11.755540 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4506bb-8dba-4159-b94e-ddcb3173c48c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:11.755959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4506bb-8dba-4159-b94e-ddcb3173c48c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:11.756237 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4506bb-8dba-4159-b94e-ddcb3173c48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:11.775604 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-ef4506bb-8dba-4159-b94e-ddcb3173c48c None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:11.776817 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ef4506bb-8dba-4159-b94e-ddcb3173c48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:11.818419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:11.837889 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1260/2490] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:39:11.848018 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afd5ed-caad-4263-bf02-a7c6eb6a66eb admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:11.849414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-74afd5ed-caad-4263-bf02-a7c6eb6a66eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:11.909613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74afd5ed-caad-4263-bf02-a7c6eb6a66eb admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:11.913248 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839b052-a004-443b-ae3c-189127564da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:11.913854 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839b052-a004-443b-ae3c-189127564da7 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:11.914134 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839b052-a004-443b-ae3c-189127564da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:11.914406 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839b052-a004-443b-ae3c-189127564da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:11.923990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839b052-a004-443b-ae3c-189127564da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:11.925419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839b052-a004-443b-ae3c-189127564da7 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:11.934733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1231/2491] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:12.206098 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4506bb-8dba-4159-b94e-ddcb3173c48c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:12.207548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-ef4506bb-8dba-4159-b94e-ddcb3173c48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:12.219662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4506bb-8dba-4159-b94e-ddcb3173c48c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:12.224338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524de7f-5775-4609-8373-20aadb47af5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:12.224980 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524de7f-5775-4609-8373-20aadb47af5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:12.225262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524de7f-5775-4609-8373-20aadb47af5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:12.225535 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524de7f-5775-4609-8373-20aadb47af5a admin admin] PATH_INFO: `/v3/projects/179f6e9bbf1d4f6bb7f57f88d5941ac7` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:12.226681 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524de7f-5775-4609-8373-20aadb47af5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=179f6e9bbf1d4f6bb7f57f88d5941ac7)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:12.228943 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524de7f-5775-4609-8373-20aadb47af5a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:12.276079 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8524de7f-5775-4609-8373-20aadb47af5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '179f6e9bbf1d4f6bb7f57f88d5941ac7', 'request_id': 'req-8524de7f-5775-4609-8373-20aadb47af5a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:12.281869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8524de7f-5775-4609-8373-20aadb47af5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '179f6e9bbf1d4f6bb7f57f88d5941ac7', 'request_id': 'req-8524de7f-5775-4609-8373-20aadb47af5a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:12.289014 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8524de7f-5775-4609-8373-20aadb47af5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '179f6e9bbf1d4f6bb7f57f88d5941ac7', 'request_id': 'req-8524de7f-5775-4609-8373-20aadb47af5a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:12.300857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8524de7f-5775-4609-8373-20aadb47af5a admin admin] The token cache is being invalidate because project 179f6e9bbf1d4f6bb7f57f88d5941ac7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:12.301270 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8524de7f-5775-4609-8373-20aadb47af5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:12.302274 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8524de7f-5775-4609-8373-20aadb47af5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:12.304855 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1261/2492] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:12 2022] DELETE /identity/v3/projects/179f6e9bbf1d4f6bb7f57f88d5941ac7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:12.310429 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cfd91-3ef9-4fc6-967e-867a35e112f9 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:12.310852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cfd91-3ef9-4fc6-967e-867a35e112f9 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:12.310999 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1cfd91-3ef9-4fc6-967e-867a35e112f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:12.331750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-aa1cfd91-3ef9-4fc6-967e-867a35e112f9 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:12.332948 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-aa1cfd91-3ef9-4fc6-967e-867a35e112f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:12.395185 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1232/2493] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:12.403831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1ed75-2e2e-4a06-b56a-e368b1cc1cd8 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:12.404818 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1ed75-2e2e-4a06-b56a-e368b1cc1cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:12.404818 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1ed75-2e2e-4a06-b56a-e368b1cc1cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:12.432677 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a1b1ed75-2e2e-4a06-b56a-e368b1cc1cd8 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:12.434053 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a1b1ed75-2e2e-4a06-b56a-e368b1cc1cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:12.445890 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1262/2494] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:12.447026 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1cfd91-3ef9-4fc6-967e-867a35e112f9 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:12.458519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1cfd91-3ef9-4fc6-967e-867a35e112f9 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:12.463523 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811761c-a3df-4c6a-aac1-d065b042e225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:12.464171 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811761c-a3df-4c6a-aac1-d065b042e225 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:12.464447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811761c-a3df-4c6a-aac1-d065b042e225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:12.464716 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811761c-a3df-4c6a-aac1-d065b042e225 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:12.468051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811761c-a3df-4c6a-aac1-d065b042e225 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:39:12.468051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811761c-a3df-4c6a-aac1-d065b042e225 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:12.468051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811761c-a3df-4c6a-aac1-d065b042e225 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:12.489316 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1233/2495] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:39:12.490301 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1ed75-2e2e-4a06-b56a-e368b1cc1cd8 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:12.500502 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1ed75-2e2e-4a06-b56a-e368b1cc1cd8 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:12.510525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bcad2-ade0-466d-ac08-c3be24327caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:12.510525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bcad2-ade0-466d-ac08-c3be24327caf admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:12.510525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bcad2-ade0-466d-ac08-c3be24327caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:12.510525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bcad2-ade0-466d-ac08-c3be24327caf admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:12.511132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593bcad2-ade0-466d-ac08-c3be24327caf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:12.512293 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593bcad2-ade0-466d-ac08-c3be24327caf admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:12.547537 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1263/2496] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:12 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:12.552230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811761c-a3df-4c6a-aac1-d065b042e225 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:12.566044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811761c-a3df-4c6a-aac1-d065b042e225 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:12.627395 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31099e9c-15be-48df-a12a-fc13ebbed29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:12.628029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31099e9c-15be-48df-a12a-fc13ebbed29f admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:12.628307 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31099e9c-15be-48df-a12a-fc13ebbed29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:12.628640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31099e9c-15be-48df-a12a-fc13ebbed29f admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:12.629256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31099e9c-15be-48df-a12a-fc13ebbed29f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:12.631276 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31099e9c-15be-48df-a12a-fc13ebbed29f admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:12.652359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1234/2497] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:12 2022] POST /identity/v3/users => generated 282 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:12.657631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bcad2-ade0-466d-ac08-c3be24327caf admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:12.668024 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593bcad2-ade0-466d-ac08-c3be24327caf admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:12.672849 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0e0c6-8963-493e-b023-92fe90b2d249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:12.673444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0e0c6-8963-493e-b023-92fe90b2d249 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:12.673725 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0e0c6-8963-493e-b023-92fe90b2d249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:12.674002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc0e0c6-8963-493e-b023-92fe90b2d249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:12.674640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0e0c6-8963-493e-b023-92fe90b2d249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:12.676311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc0e0c6-8963-493e-b023-92fe90b2d249 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:12.686515 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1264/2498] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:12.690494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31099e9c-15be-48df-a12a-fc13ebbed29f admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:12.701524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31099e9c-15be-48df-a12a-fc13ebbed29f admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:12.707187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7a49c-8449-4dd0-b055-f9dd6251b3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:12.707864 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7a49c-8449-4dd0-b055-f9dd6251b3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:12.708142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7a49c-8449-4dd0-b055-f9dd6251b3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:12.708417 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7a49c-8449-4dd0-b055-f9dd6251b3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:12.709062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7a49c-8449-4dd0-b055-f9dd6251b3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:12.710663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7a49c-8449-4dd0-b055-f9dd6251b3f3 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:12.717162 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:12.723556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1235/2499] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:12.727477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0e0c6-8963-493e-b023-92fe90b2d249 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:12.737749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc0e0c6-8963-493e-b023-92fe90b2d249 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:12.742477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130143d-b400-4ec7-878c-92c0c32faa31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:12.743180 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130143d-b400-4ec7-878c-92c0c32faa31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:12.743459 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130143d-b400-4ec7-878c-92c0c32faa31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:12.743783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130143d-b400-4ec7-878c-92c0c32faa31 admin admin] PATH_INFO: `/v3/projects/3ced185118af4090a63565bca9f75621/users/67d8a09d3d9f4a10aafec7e4b31d68b5/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:12.759448 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130143d-b400-4ec7-878c-92c0c32faa31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ced185118af4090a63565bca9f75621, user_id=67d8a09d3d9f4a10aafec7e4b31d68b5, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:12.762220 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130143d-b400-4ec7-878c-92c0c32faa31 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:12.775273 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1265/2500] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:12 2022] PUT /identity/v3/projects/3ced185118af4090a63565bca9f75621/users/67d8a09d3d9f4a10aafec7e4b31d68b5/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:12.788173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6678a-371f-48e9-a3a2-3f9f639f4292 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:12.788471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6678a-371f-48e9-a3a2-3f9f639f4292 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:12.788822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6678a-371f-48e9-a3a2-3f9f639f4292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:12.821738 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-16a6678a-371f-48e9-a3a2-3f9f639f4292 None None] MFA Rules not processed for user `67d8a09d3d9f4a10aafec7e4b31d68b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:12.822960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-16a6678a-371f-48e9-a3a2-3f9f639f4292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:12.896257 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1236/2501] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:12.900953 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130143d-b400-4ec7-878c-92c0c32faa31 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:12.965148 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130143d-b400-4ec7-878c-92c0c32faa31 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:12.970031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b48d0-a3f1-4405-bf47-789f4a40a117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:12.970637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b48d0-a3f1-4405-bf47-789f4a40a117 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:12.970915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b48d0-a3f1-4405-bf47-789f4a40a117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:12.971183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b48d0-a3f1-4405-bf47-789f4a40a117 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:12.971840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b48d0-a3f1-4405-bf47-789f4a40a117 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:12.973777 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b48d0-a3f1-4405-bf47-789f4a40a117 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:12.984890 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:13.002456 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1266/2502] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:12 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:13.006553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6678a-371f-48e9-a3a2-3f9f639f4292 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:13.017732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6678a-371f-48e9-a3a2-3f9f639f4292 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:13.071485 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755fb4f-929e-4313-b7f4-890d8de70477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:13.072110 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755fb4f-929e-4313-b7f4-890d8de70477 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:13.072388 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755fb4f-929e-4313-b7f4-890d8de70477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:13.072682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755fb4f-929e-4313-b7f4-890d8de70477 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:13.073293 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755fb4f-929e-4313-b7f4-890d8de70477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:13.076210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755fb4f-929e-4313-b7f4-890d8de70477 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:13.099538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1237/2503] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:13 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:39:13.103530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b48d0-a3f1-4405-bf47-789f4a40a117 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:13.115170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b48d0-a3f1-4405-bf47-789f4a40a117 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:13.119908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099f7c2-760b-48ee-b007-b89310800f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:13.120510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099f7c2-760b-48ee-b007-b89310800f48 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:13.120795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099f7c2-760b-48ee-b007-b89310800f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:13.121071 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099f7c2-760b-48ee-b007-b89310800f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:13.121740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099f7c2-760b-48ee-b007-b89310800f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:13.123860 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099f7c2-760b-48ee-b007-b89310800f48 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:13.134994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1267/2504] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:13.144859 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755fb4f-929e-4313-b7f4-890d8de70477 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:13.156067 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755fb4f-929e-4313-b7f4-890d8de70477 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:13.161004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f024d-0463-452e-a0c9-dc8edcef069e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:13.161658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f024d-0463-452e-a0c9-dc8edcef069e admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:13.161941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f024d-0463-452e-a0c9-dc8edcef069e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:13.162214 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f024d-0463-452e-a0c9-dc8edcef069e admin admin] PATH_INFO: `/v3/projects/9923b9017dd744bc935efc1c13199f35/users/1a6d643c531a49e1a064fa1025ed65d4/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:13.177862 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f024d-0463-452e-a0c9-dc8edcef069e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9923b9017dd744bc935efc1c13199f35, user_id=1a6d643c531a49e1a064fa1025ed65d4, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:13.180549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f024d-0463-452e-a0c9-dc8edcef069e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:13.193349 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1238/2505] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:13 2022] PUT /identity/v3/projects/9923b9017dd744bc935efc1c13199f35/users/1a6d643c531a49e1a064fa1025ed65d4/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:13.197429 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099f7c2-760b-48ee-b007-b89310800f48 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:13.207997 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099f7c2-760b-48ee-b007-b89310800f48 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:13.263886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31196f-34f6-40a9-990d-1ddd5dd4485c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:13.264426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31196f-34f6-40a9-990d-1ddd5dd4485c admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:13.264748 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31196f-34f6-40a9-990d-1ddd5dd4485c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:13.264983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31196f-34f6-40a9-990d-1ddd5dd4485c admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:13.265621 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31196f-34f6-40a9-990d-1ddd5dd4485c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:13.267219 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31196f-34f6-40a9-990d-1ddd5dd4485c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:13.276912 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1268/2506] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:13.281718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f024d-0463-452e-a0c9-dc8edcef069e admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:13.292137 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f024d-0463-452e-a0c9-dc8edcef069e admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:13.297211 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629643e5-1005-4c34-9df4-346b920fd028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:13.297859 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629643e5-1005-4c34-9df4-346b920fd028 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:13.298140 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629643e5-1005-4c34-9df4-346b920fd028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:13.298417 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629643e5-1005-4c34-9df4-346b920fd028 admin admin] PATH_INFO: `/v3/projects/9923b9017dd744bc935efc1c13199f35/users/1a6d643c531a49e1a064fa1025ed65d4/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:13.300339 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629643e5-1005-4c34-9df4-346b920fd028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9923b9017dd744bc935efc1c13199f35, user_id=1a6d643c531a49e1a064fa1025ed65d4, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:13.302961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629643e5-1005-4c34-9df4-346b920fd028 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:13.313155 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1239/2507] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:13 2022] PUT /identity/v3/projects/9923b9017dd744bc935efc1c13199f35/users/1a6d643c531a49e1a064fa1025ed65d4/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:13.324384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551b9cc-c2cc-4a81-a579-4d8497ddaa77 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:13.324678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551b9cc-c2cc-4a81-a579-4d8497ddaa77 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:13.325753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551b9cc-c2cc-4a81-a579-4d8497ddaa77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:13.353878 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e551b9cc-c2cc-4a81-a579-4d8497ddaa77 None None] MFA Rules not processed for user `1a6d643c531a49e1a064fa1025ed65d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:13.355131 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e551b9cc-c2cc-4a81-a579-4d8497ddaa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:13.422974 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1269/2508] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:13.434142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629643e5-1005-4c34-9df4-346b920fd028 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:13.435900 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-629643e5-1005-4c34-9df4-346b920fd028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:13.481991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:13.494474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629643e5-1005-4c34-9df4-346b920fd028 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:13.498161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580bcde-111b-477c-b2de-67cd3f823623 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:13.498756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580bcde-111b-477c-b2de-67cd3f823623 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:13.499035 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580bcde-111b-477c-b2de-67cd3f823623 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:13.499307 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580bcde-111b-477c-b2de-67cd3f823623 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:13.508019 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580bcde-111b-477c-b2de-67cd3f823623 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:13.509440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580bcde-111b-477c-b2de-67cd3f823623 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:13.519180 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1240/2509] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:13.688966 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551b9cc-c2cc-4a81-a579-4d8497ddaa77 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:13.690691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e551b9cc-c2cc-4a81-a579-4d8497ddaa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:13.745977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580bcde-111b-477c-b2de-67cd3f823623 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:13.747456 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d580bcde-111b-477c-b2de-67cd3f823623 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:13.769083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551b9cc-c2cc-4a81-a579-4d8497ddaa77 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:13.772741 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413e1ad-7896-4ce0-980d-a28cdf775a11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:13.773852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413e1ad-7896-4ce0-980d-a28cdf775a11 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:13.773852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413e1ad-7896-4ce0-980d-a28cdf775a11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:13.773852 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413e1ad-7896-4ce0-980d-a28cdf775a11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:13.776545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2413e1ad-7896-4ce0-980d-a28cdf775a11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:13.799870 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:13.819116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580bcde-111b-477c-b2de-67cd3f823623 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:13.827155 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125c34d-54b0-4373-93f1-0a3f89aa517b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:13.827881 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125c34d-54b0-4373-93f1-0a3f89aa517b service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:13.828161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125c34d-54b0-4373-93f1-0a3f89aa517b service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:13.828432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125c34d-54b0-4373-93f1-0a3f89aa517b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:13.830206 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d125c34d-54b0-4373-93f1-0a3f89aa517b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:13.865117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413e1ad-7896-4ce0-980d-a28cdf775a11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:13.866689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413e1ad-7896-4ce0-980d-a28cdf775a11 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:13.879337 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1270/2510] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:13.911409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d125c34d-54b0-4373-93f1-0a3f89aa517b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:13.912908 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d125c34d-54b0-4373-93f1-0a3f89aa517b service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:13.926459 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1241/2511] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:39:13.992128 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413e1ad-7896-4ce0-980d-a28cdf775a11 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:14.002182 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413e1ad-7896-4ce0-980d-a28cdf775a11 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:14.006782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b36cd2-2da6-4da3-9229-099a2449df01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:14.007400 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b36cd2-2da6-4da3-9229-099a2449df01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:14.007707 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b36cd2-2da6-4da3-9229-099a2449df01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:14.007980 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b36cd2-2da6-4da3-9229-099a2449df01 admin admin] PATH_INFO: `/v3/users/67d8a09d3d9f4a10aafec7e4b31d68b5` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:14.009094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b36cd2-2da6-4da3-9229-099a2449df01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67d8a09d3d9f4a10aafec7e4b31d68b5)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:14.011038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b36cd2-2da6-4da3-9229-099a2449df01 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:14.064323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b2b36cd2-2da6-4da3-9229-099a2449df01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67d8a09d3d9f4a10aafec7e4b31d68b5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:14.064991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b2b36cd2-2da6-4da3-9229-099a2449df01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67d8a09d3d9f4a10aafec7e4b31d68b5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:14.072392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b2b36cd2-2da6-4da3-9229-099a2449df01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67d8a09d3d9f4a10aafec7e4b31d68b5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:14.081350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b2b36cd2-2da6-4da3-9229-099a2449df01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67d8a09d3d9f4a10aafec7e4b31d68b5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:14.094158 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b2b36cd2-2da6-4da3-9229-099a2449df01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67d8a09d3d9f4a10aafec7e4b31d68b5'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:14.100369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1271/2512] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:13 2022] DELETE /identity/v3/users/67d8a09d3d9f4a10aafec7e4b31d68b5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:14.105109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb2387-58bc-47ec-94ec-0db861c6238c None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:14.105409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb2387-58bc-47ec-94ec-0db861c6238c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:14.105687 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb2387-58bc-47ec-94ec-0db861c6238c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:14.127036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-eafb2387-58bc-47ec-94ec-0db861c6238c None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:14.128291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-eafb2387-58bc-47ec-94ec-0db861c6238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:14.200805 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1242/2513] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:39:14.213423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b36cd2-2da6-4da3-9229-099a2449df01 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:14.214875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b2b36cd2-2da6-4da3-9229-099a2449df01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:14.273320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b36cd2-2da6-4da3-9229-099a2449df01 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:14.277046 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35059135-8ecc-4901-b9ab-4bca6db249e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:14.277648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35059135-8ecc-4901-b9ab-4bca6db249e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:14.277922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35059135-8ecc-4901-b9ab-4bca6db249e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:14.278899 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35059135-8ecc-4901-b9ab-4bca6db249e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:14.288053 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35059135-8ecc-4901-b9ab-4bca6db249e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:14.289517 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35059135-8ecc-4901-b9ab-4bca6db249e1 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:14.299749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1272/2514] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:14.580177 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb2387-58bc-47ec-94ec-0db861c6238c None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:14.581908 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-eafb2387-58bc-47ec-94ec-0db861c6238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:14.600120 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb2387-58bc-47ec-94ec-0db861c6238c None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:14.605992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0437493-ab71-4ff6-a42f-28cfc1212970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:14.606724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0437493-ab71-4ff6-a42f-28cfc1212970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:14.607005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0437493-ab71-4ff6-a42f-28cfc1212970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:14.607275 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0437493-ab71-4ff6-a42f-28cfc1212970 admin admin] PATH_INFO: `/v3/projects/3ced185118af4090a63565bca9f75621` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:14.615562 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0437493-ab71-4ff6-a42f-28cfc1212970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ced185118af4090a63565bca9f75621)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:14.617571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0437493-ab71-4ff6-a42f-28cfc1212970 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:14.657197 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:14.674907 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a0437493-ab71-4ff6-a42f-28cfc1212970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ced185118af4090a63565bca9f75621', 'request_id': 'req-a0437493-ab71-4ff6-a42f-28cfc1212970'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:14.681334 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a0437493-ab71-4ff6-a42f-28cfc1212970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ced185118af4090a63565bca9f75621', 'request_id': 'req-a0437493-ab71-4ff6-a42f-28cfc1212970'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:14.695265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a0437493-ab71-4ff6-a42f-28cfc1212970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ced185118af4090a63565bca9f75621', 'request_id': 'req-a0437493-ab71-4ff6-a42f-28cfc1212970'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:14.708549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a0437493-ab71-4ff6-a42f-28cfc1212970 admin admin] The token cache is being invalidate because project 3ced185118af4090a63565bca9f75621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:14.708842 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a0437493-ab71-4ff6-a42f-28cfc1212970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:14.709502 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a0437493-ab71-4ff6-a42f-28cfc1212970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:14.711202 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1243/2515] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:14 2022] DELETE /identity/v3/projects/3ced185118af4090a63565bca9f75621 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:14.715050 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35059135-8ecc-4901-b9ab-4bca6db249e1 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:14.724806 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35059135-8ecc-4901-b9ab-4bca6db249e1 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:14.750445 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:14.787846 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb3d1a-2b98-4244-8f64-c0b4018756c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:14.788515 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb3d1a-2b98-4244-8f64-c0b4018756c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:14.788797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb3d1a-2b98-4244-8f64-c0b4018756c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:14.789076 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb3d1a-2b98-4244-8f64-c0b4018756c9 admin admin] PATH_INFO: `/v3/users/1a6d643c531a49e1a064fa1025ed65d4` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:14.790207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb3d1a-2b98-4244-8f64-c0b4018756c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a6d643c531a49e1a064fa1025ed65d4)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:14.792277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb3d1a-2b98-4244-8f64-c0b4018756c9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:14.842572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-befb3d1a-2b98-4244-8f64-c0b4018756c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a6d643c531a49e1a064fa1025ed65d4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:14.843153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-befb3d1a-2b98-4244-8f64-c0b4018756c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a6d643c531a49e1a064fa1025ed65d4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:14.850250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-befb3d1a-2b98-4244-8f64-c0b4018756c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a6d643c531a49e1a064fa1025ed65d4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:14.859171 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-befb3d1a-2b98-4244-8f64-c0b4018756c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a6d643c531a49e1a064fa1025ed65d4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:14.874717 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-befb3d1a-2b98-4244-8f64-c0b4018756c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a6d643c531a49e1a064fa1025ed65d4'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:14.881436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1273/2516] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:14 2022] DELETE /identity/v3/users/1a6d643c531a49e1a064fa1025ed65d4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:15.209133 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0437493-ab71-4ff6-a42f-28cfc1212970 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:15.210729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a0437493-ab71-4ff6-a42f-28cfc1212970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:15.245111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0437493-ab71-4ff6-a42f-28cfc1212970 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:15.323583 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5e4c3-458c-486f-b704-8b916d91d01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:15.324253 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5e4c3-458c-486f-b704-8b916d91d01d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:15.324532 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5e4c3-458c-486f-b704-8b916d91d01d admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:15.325132 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5e4c3-458c-486f-b704-8b916d91d01d admin admin] PATH_INFO: `/v3/projects/9923b9017dd744bc935efc1c13199f35` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:15.326835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5e4c3-458c-486f-b704-8b916d91d01d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9923b9017dd744bc935efc1c13199f35)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:15.328733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5e4c3-458c-486f-b704-8b916d91d01d admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:15.393395 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-9fb5e4c3-458c-486f-b704-8b916d91d01d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9923b9017dd744bc935efc1c13199f35', 'request_id': 'req-9fb5e4c3-458c-486f-b704-8b916d91d01d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:15.400557 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-9fb5e4c3-458c-486f-b704-8b916d91d01d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9923b9017dd744bc935efc1c13199f35', 'request_id': 'req-9fb5e4c3-458c-486f-b704-8b916d91d01d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:15.410959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-9fb5e4c3-458c-486f-b704-8b916d91d01d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9923b9017dd744bc935efc1c13199f35', 'request_id': 'req-9fb5e4c3-458c-486f-b704-8b916d91d01d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:15.427764 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-9fb5e4c3-458c-486f-b704-8b916d91d01d admin admin] The token cache is being invalidate because project 9923b9017dd744bc935efc1c13199f35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:15.428338 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-9fb5e4c3-458c-486f-b704-8b916d91d01d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:15.428892 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-9fb5e4c3-458c-486f-b704-8b916d91d01d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:15.434543 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1244/2517] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:15 2022] DELETE /identity/v3/projects/9923b9017dd744bc935efc1c13199f35 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:15.437134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392f290-7b0e-4c2e-b558-5a020949a7d1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:15.437609 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392f290-7b0e-4c2e-b558-5a020949a7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:15.437709 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392f290-7b0e-4c2e-b558-5a020949a7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:15.466035 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b392f290-7b0e-4c2e-b558-5a020949a7d1 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:15.466657 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b392f290-7b0e-4c2e-b558-5a020949a7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:15.553767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1274/2518] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:39:15.557011 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03925014-6473-4950-9644-0f52adbdd15b None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:15.557363 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03925014-6473-4950-9644-0f52adbdd15b None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:15.557640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03925014-6473-4950-9644-0f52adbdd15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:15.580240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-03925014-6473-4950-9644-0f52adbdd15b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:15.581576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-03925014-6473-4950-9644-0f52adbdd15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:15.589863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1245/2519] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:15.594948 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392f290-7b0e-4c2e-b558-5a020949a7d1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:15.605308 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392f290-7b0e-4c2e-b558-5a020949a7d1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:15.611000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447e0c8-9e4d-4e41-970d-59b8c5c11ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:15.611615 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447e0c8-9e4d-4e41-970d-59b8c5c11ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:15.611921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447e0c8-9e4d-4e41-970d-59b8c5c11ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:15.612191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447e0c8-9e4d-4e41-970d-59b8c5c11ea3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:15.613328 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447e0c8-9e4d-4e41-970d-59b8c5c11ea3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:39:15.613328 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447e0c8-9e4d-4e41-970d-59b8c5c11ea3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:15.614745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447e0c8-9e4d-4e41-970d-59b8c5c11ea3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:15.627538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1275/2520] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:39:15.631687 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03925014-6473-4950-9644-0f52adbdd15b None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:15.640831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03925014-6473-4950-9644-0f52adbdd15b None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:15.646214 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bd1db-b766-458c-a7ae-064bb56b03f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:15.646819 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bd1db-b766-458c-a7ae-064bb56b03f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:15.647096 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bd1db-b766-458c-a7ae-064bb56b03f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:15.647366 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19bd1db-b766-458c-a7ae-064bb56b03f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:15.648016 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bd1db-b766-458c-a7ae-064bb56b03f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:15.649918 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19bd1db-b766-458c-a7ae-064bb56b03f0 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:15.682681 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1246/2521] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:15 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:15.687713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447e0c8-9e4d-4e41-970d-59b8c5c11ea3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:15.697981 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447e0c8-9e4d-4e41-970d-59b8c5c11ea3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:15.729031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:15.762987 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14efaa6-9a8b-4e7d-b196-88f66d5d4a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:15.763610 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14efaa6-9a8b-4e7d-b196-88f66d5d4a87 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:15.763925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14efaa6-9a8b-4e7d-b196-88f66d5d4a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:15.764196 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14efaa6-9a8b-4e7d-b196-88f66d5d4a87 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:15.764832 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14efaa6-9a8b-4e7d-b196-88f66d5d4a87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:15.766794 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14efaa6-9a8b-4e7d-b196-88f66d5d4a87 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:15.790814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1276/2522] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:15 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:15.795731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bd1db-b766-458c-a7ae-064bb56b03f0 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:15.807506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19bd1db-b766-458c-a7ae-064bb56b03f0 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:15.813899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c345c2-8924-47b1-b4f5-0e800794f745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:15.814646 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c345c2-8924-47b1-b4f5-0e800794f745 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:15.814995 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c345c2-8924-47b1-b4f5-0e800794f745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:15.815337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c345c2-8924-47b1-b4f5-0e800794f745 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:15.816093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c345c2-8924-47b1-b4f5-0e800794f745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:15.817763 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c345c2-8924-47b1-b4f5-0e800794f745 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:15.837206 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1247/2523] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:15.842849 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14efaa6-9a8b-4e7d-b196-88f66d5d4a87 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:15.863993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14efaa6-9a8b-4e7d-b196-88f66d5d4a87 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:15.869345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410304e-9640-4345-8c20-1b8253bac2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:15.870027 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410304e-9640-4345-8c20-1b8253bac2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:15.870315 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410304e-9640-4345-8c20-1b8253bac2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:15.870594 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410304e-9640-4345-8c20-1b8253bac2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:15.871253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410304e-9640-4345-8c20-1b8253bac2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:15.873671 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410304e-9640-4345-8c20-1b8253bac2b8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:15.892358 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1277/2524] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:15.897013 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c345c2-8924-47b1-b4f5-0e800794f745 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:15.921728 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c345c2-8924-47b1-b4f5-0e800794f745 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:15.927808 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30059dc-6b88-46a4-804f-a7ae34a02372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:15.928503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30059dc-6b88-46a4-804f-a7ae34a02372 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:15.928784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30059dc-6b88-46a4-804f-a7ae34a02372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:15.929060 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30059dc-6b88-46a4-804f-a7ae34a02372 admin admin] PATH_INFO: `/v3/projects/fb75df347cd94776983f5476d5f3614b/users/b6b64c14a36e49f6b676eae940712692/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:15.932592 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:15.948196 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30059dc-6b88-46a4-804f-a7ae34a02372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb75df347cd94776983f5476d5f3614b, user_id=b6b64c14a36e49f6b676eae940712692, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:15.952652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30059dc-6b88-46a4-804f-a7ae34a02372 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:15.981246 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1248/2525] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:15 2022] PUT /identity/v3/projects/fb75df347cd94776983f5476d5f3614b/users/b6b64c14a36e49f6b676eae940712692/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:15.997668 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d2848-a60f-408b-88ae-ac1de5a31457 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:15.998031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d2848-a60f-408b-88ae-ac1de5a31457 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:15.998312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d2848-a60f-408b-88ae-ac1de5a31457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.040390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-811d2848-a60f-408b-88ae-ac1de5a31457 None None] MFA Rules not processed for user `b6b64c14a36e49f6b676eae940712692`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:16.041680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-811d2848-a60f-408b-88ae-ac1de5a31457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:16.119153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1278/2526] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:16.124547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30059dc-6b88-46a4-804f-a7ae34a02372 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:16.191324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30059dc-6b88-46a4-804f-a7ae34a02372 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:16.196029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241e4db-8b0c-451d-8888-c6a716edd97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:16.196658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241e4db-8b0c-451d-8888-c6a716edd97a admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.196943 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241e4db-8b0c-451d-8888-c6a716edd97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.197223 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241e4db-8b0c-451d-8888-c6a716edd97a admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.197898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241e4db-8b0c-451d-8888-c6a716edd97a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:16.200224 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241e4db-8b0c-451d-8888-c6a716edd97a admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:16.230409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1249/2527] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:16 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:16.235431 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d2848-a60f-408b-88ae-ac1de5a31457 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:16.246320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d2848-a60f-408b-88ae-ac1de5a31457 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:16.308006 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00870f81-4c04-4d03-9a84-0804bc1519b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:16.308614 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00870f81-4c04-4d03-9a84-0804bc1519b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.308892 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00870f81-4c04-4d03-9a84-0804bc1519b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.309189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00870f81-4c04-4d03-9a84-0804bc1519b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.310003 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00870f81-4c04-4d03-9a84-0804bc1519b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:16.311906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00870f81-4c04-4d03-9a84-0804bc1519b6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:16.335535 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1279/2528] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:16 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:16.339479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241e4db-8b0c-451d-8888-c6a716edd97a admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:16.350791 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241e4db-8b0c-451d-8888-c6a716edd97a admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:16.355905 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7d4f5-80e5-497e-b0f1-071552d38e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:16.356912 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7d4f5-80e5-497e-b0f1-071552d38e07 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.357384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7d4f5-80e5-497e-b0f1-071552d38e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.358227 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7d4f5-80e5-497e-b0f1-071552d38e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.358905 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7d4f5-80e5-497e-b0f1-071552d38e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:16.360746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7d4f5-80e5-497e-b0f1-071552d38e07 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:16.372651 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1250/2529] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:39:16.377127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00870f81-4c04-4d03-9a84-0804bc1519b6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:16.387215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00870f81-4c04-4d03-9a84-0804bc1519b6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:16.402399 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64610d6a-fa04-4899-a0fe-d76e14bdba54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:16.403162 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64610d6a-fa04-4899-a0fe-d76e14bdba54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.404141 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64610d6a-fa04-4899-a0fe-d76e14bdba54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.404361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64610d6a-fa04-4899-a0fe-d76e14bdba54 admin admin] PATH_INFO: `/v3/projects/5e60b7594e89471eb884d051c677020b/users/391932823496448f862cb86550b0aa1d/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.426418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64610d6a-fa04-4899-a0fe-d76e14bdba54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e60b7594e89471eb884d051c677020b, user_id=391932823496448f862cb86550b0aa1d, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:16.429154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64610d6a-fa04-4899-a0fe-d76e14bdba54 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:16.444960 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1280/2530] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:16 2022] PUT /identity/v3/projects/5e60b7594e89471eb884d051c677020b/users/391932823496448f862cb86550b0aa1d/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:16.448946 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7d4f5-80e5-497e-b0f1-071552d38e07 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:16.460576 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7d4f5-80e5-497e-b0f1-071552d38e07 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:16.521937 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc493b6b-9ba2-4632-ac09-eddb643e8011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:16.522548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc493b6b-9ba2-4632-ac09-eddb643e8011 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.522831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc493b6b-9ba2-4632-ac09-eddb643e8011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.523111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc493b6b-9ba2-4632-ac09-eddb643e8011 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.523796 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc493b6b-9ba2-4632-ac09-eddb643e8011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:16.525510 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc493b6b-9ba2-4632-ac09-eddb643e8011 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:16.536455 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1251/2531] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:16.540397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64610d6a-fa04-4899-a0fe-d76e14bdba54 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:16.551674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64610d6a-fa04-4899-a0fe-d76e14bdba54 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:16.556508 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33884fd0-55d6-4bc6-aede-ce5ebacbb642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:16.557169 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33884fd0-55d6-4bc6-aede-ce5ebacbb642 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.557459 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33884fd0-55d6-4bc6-aede-ce5ebacbb642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.557738 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33884fd0-55d6-4bc6-aede-ce5ebacbb642 admin admin] PATH_INFO: `/v3/projects/5e60b7594e89471eb884d051c677020b/users/391932823496448f862cb86550b0aa1d/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.560407 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33884fd0-55d6-4bc6-aede-ce5ebacbb642 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e60b7594e89471eb884d051c677020b, user_id=391932823496448f862cb86550b0aa1d, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:16.563077 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33884fd0-55d6-4bc6-aede-ce5ebacbb642 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:16.565511 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:16.576686 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1281/2532] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:16 2022] PUT /identity/v3/projects/5e60b7594e89471eb884d051c677020b/users/391932823496448f862cb86550b0aa1d/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:16.586871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a243ae-8109-4ea2-9875-8c09a0288f6e None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.587169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a243ae-8109-4ea2-9875-8c09a0288f6e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.587449 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a243ae-8109-4ea2-9875-8c09a0288f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.615983 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-49a243ae-8109-4ea2-9875-8c09a0288f6e None None] MFA Rules not processed for user `391932823496448f862cb86550b0aa1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:16.617186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-49a243ae-8109-4ea2-9875-8c09a0288f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:16.663075 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:16.689679 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1252/2533] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:16.701599 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33884fd0-55d6-4bc6-aede-ce5ebacbb642 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:16.703326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-33884fd0-55d6-4bc6-aede-ce5ebacbb642 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:16.765971 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33884fd0-55d6-4bc6-aede-ce5ebacbb642 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:16.769698 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4265ef-491c-4fbc-a3dc-9fc395849523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:16.770310 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4265ef-491c-4fbc-a3dc-9fc395849523 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.770593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4265ef-491c-4fbc-a3dc-9fc395849523 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.770868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4265ef-491c-4fbc-a3dc-9fc395849523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.780908 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4265ef-491c-4fbc-a3dc-9fc395849523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:16.782415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4265ef-491c-4fbc-a3dc-9fc395849523 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:16.793063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1282/2534] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:16.877974 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a243ae-8109-4ea2-9875-8c09a0288f6e None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:16.883804 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a243ae-8109-4ea2-9875-8c09a0288f6e None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:16.888102 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085a60e-63c7-4516-ac83-0d313372098e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:16.888744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085a60e-63c7-4516-ac83-0d313372098e service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:16.889024 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085a60e-63c7-4516-ac83-0d313372098e service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:16.889298 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085a60e-63c7-4516-ac83-0d313372098e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:16.950637 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085a60e-63c7-4516-ac83-0d313372098e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:16.952226 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5085a60e-63c7-4516-ac83-0d313372098e service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:16.962389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1253/2535] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:19.131360 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4265ef-491c-4fbc-a3dc-9fc395849523 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:19.134603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-0b4265ef-491c-4fbc-a3dc-9fc395849523 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:19.223615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085a60e-63c7-4516-ac83-0d313372098e service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:19.226917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-5085a60e-63c7-4516-ac83-0d313372098e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:19.243002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4265ef-491c-4fbc-a3dc-9fc395849523 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:19.249384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74d0b3-246c-4d78-befa-392d7c5488cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ba45e3cff74962965b248f9f017f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:19.250017 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74d0b3-246c-4d78-befa-392d7c5488cc service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:19.250302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74d0b3-246c-4d78-befa-392d7c5488cc service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:19.250576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74d0b3-246c-4d78-befa-392d7c5488cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:19.253272 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fc74d0b3-246c-4d78-befa-392d7c5488cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:19.276754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74d0b3-246c-4d78-befa-392d7c5488cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:19.278780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74d0b3-246c-4d78-befa-392d7c5488cc service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:19.291118 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1283/2536] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:19.292873 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74d0b3-246c-4d78-befa-392d7c5488cc service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:19.295806 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fc74d0b3-246c-4d78-befa-392d7c5488cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:19.326315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085a60e-63c7-4516-ac83-0d313372098e service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:19.331877 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b5497-ff2a-45a4-93be-58f02a27b5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:19.332553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b5497-ff2a-45a4-93be-58f02a27b5e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:19.332833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b5497-ff2a-45a4-93be-58f02a27b5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:19.333409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b5497-ff2a-45a4-93be-58f02a27b5e0 admin admin] PATH_INFO: `/v3/users/ed0d97f3f08f46af851492aee6a8d0ef` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:19.334841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b5497-ff2a-45a4-93be-58f02a27b5e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed0d97f3f08f46af851492aee6a8d0ef)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:19.337389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b5497-ff2a-45a4-93be-58f02a27b5e0 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:19.389678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74d0b3-246c-4d78-befa-392d7c5488cc service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:19.395074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97503f-7c6f-4d14-9424-6d7257e42fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:19.395755 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97503f-7c6f-4d14-9424-6d7257e42fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:19.396044 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97503f-7c6f-4d14-9424-6d7257e42fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:19.396482 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97503f-7c6f-4d14-9424-6d7257e42fa3 admin admin] PATH_INFO: `/v3/users/28dac3ee2d5c4ff5ba2d15b59e57d2a7` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:19.396684 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-430b5497-ff2a-45a4-93be-58f02a27b5e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed0d97f3f08f46af851492aee6a8d0ef'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.397339 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-430b5497-ff2a-45a4-93be-58f02a27b5e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed0d97f3f08f46af851492aee6a8d0ef'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.397704 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a97503f-7c6f-4d14-9424-6d7257e42fa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28dac3ee2d5c4ff5ba2d15b59e57d2a7)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:19.397950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-430b5497-ff2a-45a4-93be-58f02a27b5e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed0d97f3f08f46af851492aee6a8d0ef'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.399871 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a97503f-7c6f-4d14-9424-6d7257e42fa3 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:19.405743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-430b5497-ff2a-45a4-93be-58f02a27b5e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed0d97f3f08f46af851492aee6a8d0ef'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.416425 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-430b5497-ff2a-45a4-93be-58f02a27b5e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed0d97f3f08f46af851492aee6a8d0ef'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.431887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:19.435253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:INSERT=1,keystone:SELECT=94,keystone:DELETE=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:19.438513 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1254/2537] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:19 2022] DELETE /identity/v3/users/ed0d97f3f08f46af851492aee6a8d0ef => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:19.458044 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a97503f-7c6f-4d14-9424-6d7257e42fa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28dac3ee2d5c4ff5ba2d15b59e57d2a7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.458724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a97503f-7c6f-4d14-9424-6d7257e42fa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28dac3ee2d5c4ff5ba2d15b59e57d2a7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.466961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a97503f-7c6f-4d14-9424-6d7257e42fa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28dac3ee2d5c4ff5ba2d15b59e57d2a7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.474151 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a97503f-7c6f-4d14-9424-6d7257e42fa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28dac3ee2d5c4ff5ba2d15b59e57d2a7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.490222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6a97503f-7c6f-4d14-9424-6d7257e42fa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28dac3ee2d5c4ff5ba2d15b59e57d2a7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.495695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1284/2538] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:19 2022] DELETE /identity/v3/users/28dac3ee2d5c4ff5ba2d15b59e57d2a7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:19.640059 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b5497-ff2a-45a4-93be-58f02a27b5e0 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:19.641803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-430b5497-ff2a-45a4-93be-58f02a27b5e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:19.708014 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97503f-7c6f-4d14-9424-6d7257e42fa3 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:19.709743 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6a97503f-7c6f-4d14-9424-6d7257e42fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:19.746058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b5497-ff2a-45a4-93be-58f02a27b5e0 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:19.790100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97503f-7c6f-4d14-9424-6d7257e42fa3 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:19.796512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6c6d1-8769-42cc-93f3-05f34a93aec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:19.798146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6c6d1-8769-42cc-93f3-05f34a93aec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:19.798146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6c6d1-8769-42cc-93f3-05f34a93aec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:19.798146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6c6d1-8769-42cc-93f3-05f34a93aec8 admin admin] PATH_INFO: `/v3/projects/b38fa406b760405f8d3004db272f1e52` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:19.799538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6c6d1-8769-42cc-93f3-05f34a93aec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b38fa406b760405f8d3004db272f1e52)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:19.801381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d6c6d1-8769-42cc-93f3-05f34a93aec8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:19.813974 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed78fc-2519-4a66-a1df-f8c682540dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:19.814605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed78fc-2519-4a66-a1df-f8c682540dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:19.814880 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed78fc-2519-4a66-a1df-f8c682540dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:19.815163 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed78fc-2519-4a66-a1df-f8c682540dfe admin admin] PATH_INFO: `/v3/projects/36bd9970f0694314b966d86f13b83d4b` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:19.816375 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed78fc-2519-4a66-a1df-f8c682540dfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=36bd9970f0694314b966d86f13b83d4b)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:19.818409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed78fc-2519-4a66-a1df-f8c682540dfe admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:19.854572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-16d6c6d1-8769-42cc-93f3-05f34a93aec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b38fa406b760405f8d3004db272f1e52', 'request_id': 'req-16d6c6d1-8769-42cc-93f3-05f34a93aec8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.861364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-16d6c6d1-8769-42cc-93f3-05f34a93aec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b38fa406b760405f8d3004db272f1e52', 'request_id': 'req-16d6c6d1-8769-42cc-93f3-05f34a93aec8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.868934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-16d6c6d1-8769-42cc-93f3-05f34a93aec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b38fa406b760405f8d3004db272f1e52', 'request_id': 'req-16d6c6d1-8769-42cc-93f3-05f34a93aec8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.870721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-74ed78fc-2519-4a66-a1df-f8c682540dfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36bd9970f0694314b966d86f13b83d4b', 'request_id': 'req-74ed78fc-2519-4a66-a1df-f8c682540dfe'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.879749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-74ed78fc-2519-4a66-a1df-f8c682540dfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36bd9970f0694314b966d86f13b83d4b', 'request_id': 'req-74ed78fc-2519-4a66-a1df-f8c682540dfe'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.886641 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-16d6c6d1-8769-42cc-93f3-05f34a93aec8 admin admin] The token cache is being invalidate because project b38fa406b760405f8d3004db272f1e52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:19.886641 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-16d6c6d1-8769-42cc-93f3-05f34a93aec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.886641 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-16d6c6d1-8769-42cc-93f3-05f34a93aec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.889990 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1285/2539] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:19 2022] DELETE /identity/v3/projects/b38fa406b760405f8d3004db272f1e52 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:19.891436 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-74ed78fc-2519-4a66-a1df-f8c682540dfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36bd9970f0694314b966d86f13b83d4b', 'request_id': 'req-74ed78fc-2519-4a66-a1df-f8c682540dfe'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.895313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d864e-0c55-4622-bc00-c3e6bf0525a9 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:19.895595 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d864e-0c55-4622-bc00-c3e6bf0525a9 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:19.895933 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d864e-0c55-4622-bc00-c3e6bf0525a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:19.904564 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-74ed78fc-2519-4a66-a1df-f8c682540dfe admin admin] The token cache is being invalidate because project 36bd9970f0694314b966d86f13b83d4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:19.904859 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-74ed78fc-2519-4a66-a1df-f8c682540dfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.905941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-74ed78fc-2519-4a66-a1df-f8c682540dfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:19.907042 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1255/2540] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:19 2022] DELETE /identity/v3/projects/36bd9970f0694314b966d86f13b83d4b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:19.911817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f2fc3-2cbe-47dc-b06c-f49619532725 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:19.912183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f2fc3-2cbe-47dc-b06c-f49619532725 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:19.912442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f2fc3-2cbe-47dc-b06c-f49619532725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:19.916540 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-726d864e-0c55-4622-bc00-c3e6bf0525a9 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:19.917764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-726d864e-0c55-4622-bc00-c3e6bf0525a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:19.939806 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-6f2f2fc3-2cbe-47dc-b06c-f49619532725 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:19.939806 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-6f2f2fc3-2cbe-47dc-b06c-f49619532725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:19.955488 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:19.975716 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:19.991764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1286/2541] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:20.001016 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b146-bcf4-469a-ba86-76e5c058c8a0 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.001228 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b146-bcf4-469a-ba86-76e5c058c8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.001485 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165b146-bcf4-469a-ba86-76e5c058c8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.020558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1256/2542] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:39:20.022051 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b165b146-bcf4-469a-ba86-76e5c058c8a0 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:20.023262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b165b146-bcf4-469a-ba86-76e5c058c8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:20.029557 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f517333-501f-4c52-9927-e768ad4629e6 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.029905 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f517333-501f-4c52-9927-e768ad4629e6 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.030194 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f517333-501f-4c52-9927-e768ad4629e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.034148 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1287/2543] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:20.038181 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165b146-bcf4-469a-ba86-76e5c058c8a0 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.048203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165b146-bcf4-469a-ba86-76e5c058c8a0 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.052502 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-9f517333-501f-4c52-9927-e768ad4629e6 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:20.053537 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398f527-c038-48ec-9a28-cba24557880a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.053984 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-9f517333-501f-4c52-9927-e768ad4629e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:20.054237 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398f527-c038-48ec-9a28-cba24557880a admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.054612 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398f527-c038-48ec-9a28-cba24557880a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.054856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398f527-c038-48ec-9a28-cba24557880a admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.055506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398f527-c038-48ec-9a28-cba24557880a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:39:20.055844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398f527-c038-48ec-9a28-cba24557880a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.057563 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7398f527-c038-48ec-9a28-cba24557880a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.067032 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1257/2544] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:39:20.069027 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f517333-501f-4c52-9927-e768ad4629e6 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.074085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1288/2545] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:39:20.075983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398f527-c038-48ec-9a28-cba24557880a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.079764 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f517333-501f-4c52-9927-e768ad4629e6 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.087139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f939b-aa12-4222-bc31-325a221034bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.087655 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f939b-aa12-4222-bc31-325a221034bb admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.087947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f939b-aa12-4222-bc31-325a221034bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.088469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f939b-aa12-4222-bc31-325a221034bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.089050 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f939b-aa12-4222-bc31-325a221034bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:39:20.089251 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398f527-c038-48ec-9a28-cba24557880a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.089356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f939b-aa12-4222-bc31-325a221034bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.091898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f939b-aa12-4222-bc31-325a221034bb admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.098746 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2597f-bf47-4972-a662-5ae7e1e66faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.099508 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2597f-bf47-4972-a662-5ae7e1e66faa admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.099822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2597f-bf47-4972-a662-5ae7e1e66faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.100097 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2597f-bf47-4972-a662-5ae7e1e66faa admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.100721 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2597f-bf47-4972-a662-5ae7e1e66faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.103594 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2597f-bf47-4972-a662-5ae7e1e66faa admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.107699 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1258/2546] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:39:20.112067 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f939b-aa12-4222-bc31-325a221034bb admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.127754 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f939b-aa12-4222-bc31-325a221034bb admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.132811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267c6b8-e862-42f3-ad93-e5dc948364b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.133474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267c6b8-e862-42f3-ad93-e5dc948364b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.133772 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267c6b8-e862-42f3-ad93-e5dc948364b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.134187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267c6b8-e862-42f3-ad93-e5dc948364b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.138358 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267c6b8-e862-42f3-ad93-e5dc948364b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.138530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1289/2547] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:20 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:20.138971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267c6b8-e862-42f3-ad93-e5dc948364b2 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.139243 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2597f-bf47-4972-a662-5ae7e1e66faa admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.168392 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1259/2548] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:20 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:39:20.173010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267c6b8-e862-42f3-ad93-e5dc948364b2 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.216770 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2597f-bf47-4972-a662-5ae7e1e66faa admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.242607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267c6b8-e862-42f3-ad93-e5dc948364b2 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.248740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06727768-4d97-478f-8d1e-38fa9acd9de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.249619 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06727768-4d97-478f-8d1e-38fa9acd9de6 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.249865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06727768-4d97-478f-8d1e-38fa9acd9de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.250149 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06727768-4d97-478f-8d1e-38fa9acd9de6 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.250893 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06727768-4d97-478f-8d1e-38fa9acd9de6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.253074 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06727768-4d97-478f-8d1e-38fa9acd9de6 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.280171 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1260/2549] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:20 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:39:20.283960 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06727768-4d97-478f-8d1e-38fa9acd9de6 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.296268 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8dbde-9a0a-433c-8cbf-1c2ef576605d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.297004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8dbde-9a0a-433c-8cbf-1c2ef576605d admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.297279 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8dbde-9a0a-433c-8cbf-1c2ef576605d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.297602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8dbde-9a0a-433c-8cbf-1c2ef576605d admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.298274 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8dbde-9a0a-433c-8cbf-1c2ef576605d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.299419 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06727768-4d97-478f-8d1e-38fa9acd9de6 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.300334 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff8dbde-9a0a-433c-8cbf-1c2ef576605d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.304954 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7736a44-a276-415d-9ef0-9dd48c413e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.305654 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7736a44-a276-415d-9ef0-9dd48c413e28 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.306043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7736a44-a276-415d-9ef0-9dd48c413e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.306608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7736a44-a276-415d-9ef0-9dd48c413e28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.307950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7736a44-a276-415d-9ef0-9dd48c413e28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.310038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7736a44-a276-415d-9ef0-9dd48c413e28 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.327789 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1261/2550] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:39:20.328008 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1290/2551] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:20 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:20.329614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7736a44-a276-415d-9ef0-9dd48c413e28 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.331461 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8dbde-9a0a-433c-8cbf-1c2ef576605d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.341001 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7736a44-a276-415d-9ef0-9dd48c413e28 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.345147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8dbde-9a0a-433c-8cbf-1c2ef576605d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.346818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d65fb-3670-426e-8be2-4779ac5bb3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.347282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d65fb-3670-426e-8be2-4779ac5bb3ac admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.347572 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d65fb-3670-426e-8be2-4779ac5bb3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.347886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d65fb-3670-426e-8be2-4779ac5bb3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.348654 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d65fb-3670-426e-8be2-4779ac5bb3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.350667 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d65fb-3670-426e-8be2-4779ac5bb3ac admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.351610 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f20b0-1dfb-4c4a-9778-a50aa4e56e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.352458 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f20b0-1dfb-4c4a-9778-a50aa4e56e45 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.352774 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f20b0-1dfb-4c4a-9778-a50aa4e56e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.353049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f20b0-1dfb-4c4a-9778-a50aa4e56e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.353702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f20b0-1dfb-4c4a-9778-a50aa4e56e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.355856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f20b0-1dfb-4c4a-9778-a50aa4e56e45 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.368203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1262/2552] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:20.369230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d65fb-3670-426e-8be2-4779ac5bb3ac admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.371761 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1291/2553] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:39:20.375609 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f20b0-1dfb-4c4a-9778-a50aa4e56e45 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.380000 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d65fb-3670-426e-8be2-4779ac5bb3ac admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.384893 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d004da-071a-4203-86d0-626aab587560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.385546 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d004da-071a-4203-86d0-626aab587560 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.385831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d004da-071a-4203-86d0-626aab587560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.386102 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d004da-071a-4203-86d0-626aab587560 admin admin] PATH_INFO: `/v3/projects/e76180eb30b14ba990d9cff62a6ca5cd/users/5936ba1a669448c49db8e7e9dff51f74/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.386797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f20b0-1dfb-4c4a-9778-a50aa4e56e45 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.393585 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416753a-9100-40e8-8abc-12616b9dbed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.394201 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416753a-9100-40e8-8abc-12616b9dbed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.395079 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416753a-9100-40e8-8abc-12616b9dbed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.395079 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416753a-9100-40e8-8abc-12616b9dbed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.395426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416753a-9100-40e8-8abc-12616b9dbed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.397193 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416753a-9100-40e8-8abc-12616b9dbed2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.412171 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d004da-071a-4203-86d0-626aab587560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e76180eb30b14ba990d9cff62a6ca5cd, user_id=5936ba1a669448c49db8e7e9dff51f74, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.414922 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d004da-071a-4203-86d0-626aab587560 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.416572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1292/2554] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:39:20.421600 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416753a-9100-40e8-8abc-12616b9dbed2 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.432722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416753a-9100-40e8-8abc-12616b9dbed2 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.439140 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:20.439713 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1263/2555] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:20 2022] PUT /identity/v3/projects/e76180eb30b14ba990d9cff62a6ca5cd/users/5936ba1a669448c49db8e7e9dff51f74/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:20.447844 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b769435-af47-454c-ae11-8e3de85c5b47 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.448660 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b769435-af47-454c-ae11-8e3de85c5b47 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.448999 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b769435-af47-454c-ae11-8e3de85c5b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.482290 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-3b769435-af47-454c-ae11-8e3de85c5b47 None None] MFA Rules not processed for user `5936ba1a669448c49db8e7e9dff51f74`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:20.483524 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-3b769435-af47-454c-ae11-8e3de85c5b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:20.509102 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc0a07-f848-4400-981a-bf968b23ba6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.509705 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc0a07-f848-4400-981a-bf968b23ba6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.510344 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc0a07-f848-4400-981a-bf968b23ba6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.510344 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc0a07-f848-4400-981a-bf968b23ba6d admin admin] PATH_INFO: `/v3/projects/9ff095a5b52f4f7088f0331072c0d2f6/users/5d82dc4020f84eb6bb0ae5ccbafae68a/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.517549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:20.528418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc0a07-f848-4400-981a-bf968b23ba6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ff095a5b52f4f7088f0331072c0d2f6, user_id=5d82dc4020f84eb6bb0ae5ccbafae68a, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.531337 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc0a07-f848-4400-981a-bf968b23ba6d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.546105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1293/2556] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:20 2022] PUT /identity/v3/projects/9ff095a5b52f4f7088f0331072c0d2f6/users/5d82dc4020f84eb6bb0ae5ccbafae68a/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:20.556374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6c7c7-6dbf-41e2-9501-a80d99fe8ea1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.556682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6c7c7-6dbf-41e2-9501-a80d99fe8ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.556934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6c7c7-6dbf-41e2-9501-a80d99fe8ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.563747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1264/2557] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:39:20.576680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b769435-af47-454c-ae11-8e3de85c5b47 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.578214 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-3b769435-af47-454c-ae11-8e3de85c5b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:20.595069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6cd6c7c7-6dbf-41e2-9501-a80d99fe8ea1 None None] MFA Rules not processed for user `5d82dc4020f84eb6bb0ae5ccbafae68a`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:20.596628 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6cd6c7c7-6dbf-41e2-9501-a80d99fe8ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:20.644921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b769435-af47-454c-ae11-8e3de85c5b47 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.677229 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1294/2558] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:20.682574 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6c7c7-6dbf-41e2-9501-a80d99fe8ea1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.757141 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6c7c7-6dbf-41e2-9501-a80d99fe8ea1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.762720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c2d6d-dba6-4aba-bebe-cfa8f2c356e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.763818 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c2d6d-dba6-4aba-bebe-cfa8f2c356e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.764143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c2d6d-dba6-4aba-bebe-cfa8f2c356e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.764422 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c2d6d-dba6-4aba-bebe-cfa8f2c356e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.765073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c2d6d-dba6-4aba-bebe-cfa8f2c356e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.767056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c2d6d-dba6-4aba-bebe-cfa8f2c356e4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.800029 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1295/2559] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:20 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:39:20.805337 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c2d6d-dba6-4aba-bebe-cfa8f2c356e4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.814872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:20.823582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c64a8-8154-4473-827a-46c29479dda8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.824202 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c64a8-8154-4473-827a-46c29479dda8 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.824475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c64a8-8154-4473-827a-46c29479dda8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.825314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c64a8-8154-4473-827a-46c29479dda8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.870602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c2d6d-dba6-4aba-bebe-cfa8f2c356e4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.876088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb40f7-16e9-40ec-bc12-050aa5a6d050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.876715 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb40f7-16e9-40ec-bc12-050aa5a6d050 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.877125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb40f7-16e9-40ec-bc12-050aa5a6d050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.877410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb40f7-16e9-40ec-bc12-050aa5a6d050 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.878036 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb40f7-16e9-40ec-bc12-050aa5a6d050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.880037 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb40f7-16e9-40ec-bc12-050aa5a6d050 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.886782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c64a8-8154-4473-827a-46c29479dda8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.888991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c64a8-8154-4473-827a-46c29479dda8 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.898990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1265/2560] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:20.901894 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1296/2561] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:20 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:20.906256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c64a8-8154-4473-827a-46c29479dda8 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.916610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c64a8-8154-4473-827a-46c29479dda8 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.921483 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46771a-7dde-40e4-9268-66244ba1e932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.922073 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46771a-7dde-40e4-9268-66244ba1e932 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.922348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46771a-7dde-40e4-9268-66244ba1e932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.922640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46771a-7dde-40e4-9268-66244ba1e932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.923271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46771a-7dde-40e4-9268-66244ba1e932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.924865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46771a-7dde-40e4-9268-66244ba1e932 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.934958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1266/2562] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:39:20.939423 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb40f7-16e9-40ec-bc12-050aa5a6d050 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:20.950368 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb40f7-16e9-40ec-bc12-050aa5a6d050 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:20.955044 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c366450-a4ed-4563-924e-d02d447f087a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:20.955725 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c366450-a4ed-4563-924e-d02d447f087a admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:20.956011 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c366450-a4ed-4563-924e-d02d447f087a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:20.956295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c366450-a4ed-4563-924e-d02d447f087a admin admin] PATH_INFO: `/v3/projects/e63b32aebb7046d19116727d076c7904/users/4fc5431754cb476bade9d7a1dfbabe9e/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:20.972786 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c366450-a4ed-4563-924e-d02d447f087a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e63b32aebb7046d19116727d076c7904, user_id=4fc5431754cb476bade9d7a1dfbabe9e, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:20.975445 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c366450-a4ed-4563-924e-d02d447f087a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:20.990174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1297/2563] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:20 2022] PUT /identity/v3/projects/e63b32aebb7046d19116727d076c7904/users/4fc5431754cb476bade9d7a1dfbabe9e/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:20.994172 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46771a-7dde-40e4-9268-66244ba1e932 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:21.005006 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46771a-7dde-40e4-9268-66244ba1e932 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:21.031432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:21.066142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24595470-3e7b-491a-a4bb-22dd0dc257bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:21.066803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24595470-3e7b-491a-a4bb-22dd0dc257bf admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:21.067036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24595470-3e7b-491a-a4bb-22dd0dc257bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:21.067315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24595470-3e7b-491a-a4bb-22dd0dc257bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:21.067990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24595470-3e7b-491a-a4bb-22dd0dc257bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:21.069626 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24595470-3e7b-491a-a4bb-22dd0dc257bf admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:21.081138 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1267/2564] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:21.085184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c366450-a4ed-4563-924e-d02d447f087a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:21.096720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c366450-a4ed-4563-924e-d02d447f087a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:21.101953 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32bb38-d781-4819-989d-218d7757d75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:21.102977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32bb38-d781-4819-989d-218d7757d75d admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:21.103261 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32bb38-d781-4819-989d-218d7757d75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:21.103540 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32bb38-d781-4819-989d-218d7757d75d admin admin] PATH_INFO: `/v3/projects/e63b32aebb7046d19116727d076c7904/users/4fc5431754cb476bade9d7a1dfbabe9e/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:21.105588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32bb38-d781-4819-989d-218d7757d75d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e63b32aebb7046d19116727d076c7904, user_id=4fc5431754cb476bade9d7a1dfbabe9e, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:21.109228 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32bb38-d781-4819-989d-218d7757d75d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:21.125454 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1298/2565] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:21 2022] PUT /identity/v3/projects/e63b32aebb7046d19116727d076c7904/users/4fc5431754cb476bade9d7a1dfbabe9e/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:21.135870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a24cc-5798-4ccd-b0fc-6c85499aad8a None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:21.136275 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a24cc-5798-4ccd-b0fc-6c85499aad8a None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:21.136695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a24cc-5798-4ccd-b0fc-6c85499aad8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:21.167645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-b61a24cc-5798-4ccd-b0fc-6c85499aad8a None None] MFA Rules not processed for user `4fc5431754cb476bade9d7a1dfbabe9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:21.169561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b61a24cc-5798-4ccd-b0fc-6c85499aad8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:21.264841 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1268/2566] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:21.274834 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32bb38-d781-4819-989d-218d7757d75d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:21.334875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32bb38-d781-4819-989d-218d7757d75d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:21.338559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab02374-1a34-4982-98da-922c45748b68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:21.339177 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab02374-1a34-4982-98da-922c45748b68 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:21.339445 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab02374-1a34-4982-98da-922c45748b68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:21.339750 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab02374-1a34-4982-98da-922c45748b68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:21.398815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab02374-1a34-4982-98da-922c45748b68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:21.400305 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab02374-1a34-4982-98da-922c45748b68 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:21.409527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1299/2567] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:31.257323 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=51 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:31.405365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:32.263946 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a24cc-5798-4ccd-b0fc-6c85499aad8a None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:32.265441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b61a24cc-5798-4ccd-b0fc-6c85499aad8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:32.278729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a24cc-5798-4ccd-b0fc-6c85499aad8a None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:32.282597 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e21d61-23f0-4acf-a991-064bc652379b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:32.283225 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e21d61-23f0-4acf-a991-064bc652379b service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:32.283552 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e21d61-23f0-4acf-a991-064bc652379b service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:32.283909 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e21d61-23f0-4acf-a991-064bc652379b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:32.294617 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e21d61-23f0-4acf-a991-064bc652379b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:32.296431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e21d61-23f0-4acf-a991-064bc652379b service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:32.306506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1269/2568] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:38.509840 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab02374-1a34-4982-98da-922c45748b68 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:38.511349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9ab02374-1a34-4982-98da-922c45748b68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:38.580766 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab02374-1a34-4982-98da-922c45748b68 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:38.585324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a04ac6-4276-4a82-8c25-e073fdb5cb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:38.585951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a04ac6-4276-4a82-8c25-e073fdb5cb9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:38.586252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a04ac6-4276-4a82-8c25-e073fdb5cb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:38.586528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a04ac6-4276-4a82-8c25-e073fdb5cb9b admin admin] PATH_INFO: `/v3/users/b6b64c14a36e49f6b676eae940712692` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:38.587686 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a04ac6-4276-4a82-8c25-e073fdb5cb9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6b64c14a36e49f6b676eae940712692)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:38.589768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a04ac6-4276-4a82-8c25-e073fdb5cb9b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:38.635038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-87a04ac6-4276-4a82-8c25-e073fdb5cb9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6b64c14a36e49f6b676eae940712692'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:38.635706 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-87a04ac6-4276-4a82-8c25-e073fdb5cb9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6b64c14a36e49f6b676eae940712692'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:38.643582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-87a04ac6-4276-4a82-8c25-e073fdb5cb9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6b64c14a36e49f6b676eae940712692'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:38.651339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-87a04ac6-4276-4a82-8c25-e073fdb5cb9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6b64c14a36e49f6b676eae940712692'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:38.663609 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-87a04ac6-4276-4a82-8c25-e073fdb5cb9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6b64c14a36e49f6b676eae940712692'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:38.669705 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1300/2569] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:38 2022] DELETE /identity/v3/users/b6b64c14a36e49f6b676eae940712692 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:38.674256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e3671-402a-41a3-804e-1a14900dc3e5 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:38.674740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e3671-402a-41a3-804e-1a14900dc3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:38.674837 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e3671-402a-41a3-804e-1a14900dc3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:38.693602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-2b9e3671-402a-41a3-804e-1a14900dc3e5 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:38.694782 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2b9e3671-402a-41a3-804e-1a14900dc3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:38.751986 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1270/2570] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:38.761513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a04ac6-4276-4a82-8c25-e073fdb5cb9b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:38.762930 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-87a04ac6-4276-4a82-8c25-e073fdb5cb9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:38.780415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e3671-402a-41a3-804e-1a14900dc3e5 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:38.781811 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2b9e3671-402a-41a3-804e-1a14900dc3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:38.793465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9e3671-402a-41a3-804e-1a14900dc3e5 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:38.798835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30a2b7-d7b5-4838-aab4-fcabf2ccbbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:38.799459 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30a2b7-d7b5-4838-aab4-fcabf2ccbbc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:38.799771 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30a2b7-d7b5-4838-aab4-fcabf2ccbbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:38.800046 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30a2b7-d7b5-4838-aab4-fcabf2ccbbc6 admin admin] PATH_INFO: `/v3/users/5936ba1a669448c49db8e7e9dff51f74` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:38.801166 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30a2b7-d7b5-4838-aab4-fcabf2ccbbc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5936ba1a669448c49db8e7e9dff51f74)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:38.803074 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c30a2b7-d7b5-4838-aab4-fcabf2ccbbc6 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:38.824386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a04ac6-4276-4a82-8c25-e073fdb5cb9b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:38.828831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99cb02-be70-4893-8e71-193f6043d16e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:38.829545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99cb02-be70-4893-8e71-193f6043d16e service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:38.829866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99cb02-be70-4893-8e71-193f6043d16e service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:38.830215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99cb02-be70-4893-8e71-193f6043d16e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:38.840339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99cb02-be70-4893-8e71-193f6043d16e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:38.842560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99cb02-be70-4893-8e71-193f6043d16e service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:38.851184 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2c30a2b7-d7b5-4838-aab4-fcabf2ccbbc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5936ba1a669448c49db8e7e9dff51f74'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:38.851935 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2c30a2b7-d7b5-4838-aab4-fcabf2ccbbc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5936ba1a669448c49db8e7e9dff51f74'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:38.852503 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1301/2571] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:38.852679 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2c30a2b7-d7b5-4838-aab4-fcabf2ccbbc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5936ba1a669448c49db8e7e9dff51f74'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:38.860078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2c30a2b7-d7b5-4838-aab4-fcabf2ccbbc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5936ba1a669448c49db8e7e9dff51f74'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:38.867932 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2c30a2b7-d7b5-4838-aab4-fcabf2ccbbc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5936ba1a669448c49db8e7e9dff51f74'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:38.887208 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1271/2572] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:38 2022] DELETE /identity/v3/users/5936ba1a669448c49db8e7e9dff51f74 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:38.892319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918eabd-c06f-411f-b94e-cb3cc9870ad7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:38.892606 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918eabd-c06f-411f-b94e-cb3cc9870ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:38.892877 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918eabd-c06f-411f-b94e-cb3cc9870ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:38.913160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6918eabd-c06f-411f-b94e-cb3cc9870ad7 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:38.914512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6918eabd-c06f-411f-b94e-cb3cc9870ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:38.932922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:38.981623 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1302/2573] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:38.992936 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30a2b7-d7b5-4838-aab4-fcabf2ccbbc6 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:38.994473 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2c30a2b7-d7b5-4838-aab4-fcabf2ccbbc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:39.061881 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30a2b7-d7b5-4838-aab4-fcabf2ccbbc6 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:39.065540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bdd96-0256-40cb-bdf9-98322969196c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:39.066233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bdd96-0256-40cb-bdf9-98322969196c service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:39.066582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bdd96-0256-40cb-bdf9-98322969196c service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:39.066863 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46bdd96-0256-40cb-bdf9-98322969196c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:39.076768 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46bdd96-0256-40cb-bdf9-98322969196c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:39.078230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46bdd96-0256-40cb-bdf9-98322969196c service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:39.087966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1272/2574] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:39.169967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918eabd-c06f-411f-b94e-cb3cc9870ad7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:39.171546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6918eabd-c06f-411f-b94e-cb3cc9870ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:39.184361 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918eabd-c06f-411f-b94e-cb3cc9870ad7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:39.191922 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b9960-12bc-4f0a-9073-ae1d353993e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:39.192579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b9960-12bc-4f0a-9073-ae1d353993e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:39.192866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b9960-12bc-4f0a-9073-ae1d353993e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:39.193147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b9960-12bc-4f0a-9073-ae1d353993e0 admin admin] PATH_INFO: `/v3/projects/fb75df347cd94776983f5476d5f3614b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:39.194313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b9960-12bc-4f0a-9073-ae1d353993e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb75df347cd94776983f5476d5f3614b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:39.196400 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b9960-12bc-4f0a-9073-ae1d353993e0 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:39.249062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b41b9960-12bc-4f0a-9073-ae1d353993e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb75df347cd94776983f5476d5f3614b', 'request_id': 'req-b41b9960-12bc-4f0a-9073-ae1d353993e0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.255277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b41b9960-12bc-4f0a-9073-ae1d353993e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb75df347cd94776983f5476d5f3614b', 'request_id': 'req-b41b9960-12bc-4f0a-9073-ae1d353993e0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.262618 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b41b9960-12bc-4f0a-9073-ae1d353993e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb75df347cd94776983f5476d5f3614b', 'request_id': 'req-b41b9960-12bc-4f0a-9073-ae1d353993e0'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.274414 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b41b9960-12bc-4f0a-9073-ae1d353993e0 admin admin] The token cache is being invalidate because project fb75df347cd94776983f5476d5f3614b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:39.274962 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b41b9960-12bc-4f0a-9073-ae1d353993e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.275802 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b41b9960-12bc-4f0a-9073-ae1d353993e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.277606 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1303/2575] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:39 2022] DELETE /identity/v3/projects/fb75df347cd94776983f5476d5f3614b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:39.281823 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bdd96-0256-40cb-bdf9-98322969196c service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:39.287662 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:39.293872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46bdd96-0256-40cb-bdf9-98322969196c service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:39.358217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18894995-e7ad-4452-8117-5b574a4d6a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:39.358917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18894995-e7ad-4452-8117-5b574a4d6a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:39.359308 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18894995-e7ad-4452-8117-5b574a4d6a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:39.360202 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18894995-e7ad-4452-8117-5b574a4d6a4d admin admin] PATH_INFO: `/v3/users/391932823496448f862cb86550b0aa1d` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:39.361106 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18894995-e7ad-4452-8117-5b574a4d6a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=391932823496448f862cb86550b0aa1d)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:39.363153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18894995-e7ad-4452-8117-5b574a4d6a4d admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:39.417091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-18894995-e7ad-4452-8117-5b574a4d6a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '391932823496448f862cb86550b0aa1d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.417711 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-18894995-e7ad-4452-8117-5b574a4d6a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '391932823496448f862cb86550b0aa1d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.418314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-18894995-e7ad-4452-8117-5b574a4d6a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '391932823496448f862cb86550b0aa1d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.426285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-18894995-e7ad-4452-8117-5b574a4d6a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '391932823496448f862cb86550b0aa1d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.434038 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-18894995-e7ad-4452-8117-5b574a4d6a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '391932823496448f862cb86550b0aa1d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.439940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b9960-12bc-4f0a-9073-ae1d353993e0 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:39.441439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b41b9960-12bc-4f0a-9073-ae1d353993e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:39.455866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b9960-12bc-4f0a-9073-ae1d353993e0 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:39.458747 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1273/2576] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:39 2022] DELETE /identity/v3/users/391932823496448f862cb86550b0aa1d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:39.468240 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b9204-5a24-4c91-a388-6a49b6822fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:39.468831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b9204-5a24-4c91-a388-6a49b6822fca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:39.469109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b9204-5a24-4c91-a388-6a49b6822fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:39.469426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b9204-5a24-4c91-a388-6a49b6822fca admin admin] PATH_INFO: `/v3/projects/e76180eb30b14ba990d9cff62a6ca5cd` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:39.470556 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b9204-5a24-4c91-a388-6a49b6822fca admin admin] RBAC: Authorizing `identity:delete_project(project_id=e76180eb30b14ba990d9cff62a6ca5cd)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:39.472678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b9204-5a24-4c91-a388-6a49b6822fca admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:39.525334 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-356b9204-5a24-4c91-a388-6a49b6822fca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e76180eb30b14ba990d9cff62a6ca5cd', 'request_id': 'req-356b9204-5a24-4c91-a388-6a49b6822fca'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.533660 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-356b9204-5a24-4c91-a388-6a49b6822fca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e76180eb30b14ba990d9cff62a6ca5cd', 'request_id': 'req-356b9204-5a24-4c91-a388-6a49b6822fca'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.541014 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-356b9204-5a24-4c91-a388-6a49b6822fca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e76180eb30b14ba990d9cff62a6ca5cd', 'request_id': 'req-356b9204-5a24-4c91-a388-6a49b6822fca'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.553396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-356b9204-5a24-4c91-a388-6a49b6822fca admin admin] The token cache is being invalidate because project e76180eb30b14ba990d9cff62a6ca5cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:39.553694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-356b9204-5a24-4c91-a388-6a49b6822fca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.554452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-356b9204-5a24-4c91-a388-6a49b6822fca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.556027 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1304/2577] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:39 2022] DELETE /identity/v3/projects/e76180eb30b14ba990d9cff62a6ca5cd => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:39.562388 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695deea4-bb16-43eb-aba1-59e628e3f7c7 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:39.562698 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695deea4-bb16-43eb-aba1-59e628e3f7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:39.562903 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695deea4-bb16-43eb-aba1-59e628e3f7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:39.583029 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-695deea4-bb16-43eb-aba1-59e628e3f7c7 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:39.584277 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-695deea4-bb16-43eb-aba1-59e628e3f7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:39.653026 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1274/2578] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:39.660997 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f639a-d407-4812-9950-288a8f44fbb1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:39.661301 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f639a-d407-4812-9950-288a8f44fbb1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:39.661574 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f639a-d407-4812-9950-288a8f44fbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:39.681862 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-c10f639a-d407-4812-9950-288a8f44fbb1 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:39.682993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c10f639a-d407-4812-9950-288a8f44fbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:39.692299 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1305/2579] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:39.696551 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695deea4-bb16-43eb-aba1-59e628e3f7c7 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:39.709204 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695deea4-bb16-43eb-aba1-59e628e3f7c7 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:39.713799 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eea1a3-f5bb-4664-b515-629bd080687c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:39.714406 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eea1a3-f5bb-4664-b515-629bd080687c admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:39.714971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eea1a3-f5bb-4664-b515-629bd080687c admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:39.714971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eea1a3-f5bb-4664-b515-629bd080687c admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:39.715591 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eea1a3-f5bb-4664-b515-629bd080687c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:39:39.715971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eea1a3-f5bb-4664-b515-629bd080687c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:39.717867 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eea1a3-f5bb-4664-b515-629bd080687c admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:39.729545 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1275/2580] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:39:39.734705 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f639a-d407-4812-9950-288a8f44fbb1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:39.744680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f639a-d407-4812-9950-288a8f44fbb1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:39.749777 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332f4e1-4ddf-4582-b9fd-0ee73d5b64e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:39.750149 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eea1a3-f5bb-4664-b515-629bd080687c admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:39.750387 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332f4e1-4ddf-4582-b9fd-0ee73d5b64e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:39.750685 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332f4e1-4ddf-4582-b9fd-0ee73d5b64e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:39.750939 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332f4e1-4ddf-4582-b9fd-0ee73d5b64e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:39.751565 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332f4e1-4ddf-4582-b9fd-0ee73d5b64e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:39.752159 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-04eea1a3-f5bb-4664-b515-629bd080687c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:39.753491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332f4e1-4ddf-4582-b9fd-0ee73d5b64e4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:39.766481 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eea1a3-f5bb-4664-b515-629bd080687c admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:39.772031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702bf704-f4f6-499e-9de2-a0ed72fc13cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:39.772715 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bf704-f4f6-499e-9de2-a0ed72fc13cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:39.773229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bf704-f4f6-499e-9de2-a0ed72fc13cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:39.773500 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702bf704-f4f6-499e-9de2-a0ed72fc13cd admin admin] PATH_INFO: `/v3/projects/5e60b7594e89471eb884d051c677020b` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:39.775139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702bf704-f4f6-499e-9de2-a0ed72fc13cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e60b7594e89471eb884d051c677020b)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:39.777172 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702bf704-f4f6-499e-9de2-a0ed72fc13cd admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:39.780176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:39.784034 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:39.796479 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1306/2581] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:39 2022] POST /identity/v3/projects => generated 391 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:39.797465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332f4e1-4ddf-4582-b9fd-0ee73d5b64e4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:39.830656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-702bf704-f4f6-499e-9de2-a0ed72fc13cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e60b7594e89471eb884d051c677020b', 'request_id': 'req-702bf704-f4f6-499e-9de2-a0ed72fc13cd'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.836671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-702bf704-f4f6-499e-9de2-a0ed72fc13cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e60b7594e89471eb884d051c677020b', 'request_id': 'req-702bf704-f4f6-499e-9de2-a0ed72fc13cd'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.845298 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-702bf704-f4f6-499e-9de2-a0ed72fc13cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e60b7594e89471eb884d051c677020b', 'request_id': 'req-702bf704-f4f6-499e-9de2-a0ed72fc13cd'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.858096 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-702bf704-f4f6-499e-9de2-a0ed72fc13cd admin admin] The token cache is being invalidate because project 5e60b7594e89471eb884d051c677020b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:39.858381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-702bf704-f4f6-499e-9de2-a0ed72fc13cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.859031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-702bf704-f4f6-499e-9de2-a0ed72fc13cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:39.860609 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1276/2582] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:39 2022] DELETE /identity/v3/projects/5e60b7594e89471eb884d051c677020b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:39.865317 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda5424-7ad3-4f3c-9f8b-3364495cf45d None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:39.865596 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda5424-7ad3-4f3c-9f8b-3364495cf45d None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:39.865869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda5424-7ad3-4f3c-9f8b-3364495cf45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:39.865978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332f4e1-4ddf-4582-b9fd-0ee73d5b64e4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:39.885798 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-1eda5424-7ad3-4f3c-9f8b-3364495cf45d None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:39.886998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1eda5424-7ad3-4f3c-9f8b-3364495cf45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:39.926336 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e1cec-bae3-414c-8576-38b5d46b5b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:39.927246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e1cec-bae3-414c-8576-38b5d46b5b53 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:39.927533 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e1cec-bae3-414c-8576-38b5d46b5b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:39.927843 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e1cec-bae3-414c-8576-38b5d46b5b53 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:39.928470 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e1cec-bae3-414c-8576-38b5d46b5b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:39.930425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e1cec-bae3-414c-8576-38b5d46b5b53 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:39.956639 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1277/2583] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:39:39.956819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1307/2584] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:39 2022] POST /identity/v3/users => generated 296 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:39:39.956908 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda5424-7ad3-4f3c-9f8b-3364495cf45d None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:39.956908 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1eda5424-7ad3-4f3c-9f8b-3364495cf45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:39.957966 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3a9c4-49d7-45db-8ae9-d46fbc575a21 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:39.958253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3a9c4-49d7-45db-8ae9-d46fbc575a21 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:39.958529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3a9c4-49d7-45db-8ae9-d46fbc575a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:39.969236 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda5424-7ad3-4f3c-9f8b-3364495cf45d None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:39.973773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b7375-1c0e-4815-9c4b-87ff9a711bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:39.974367 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b7375-1c0e-4815-9c4b-87ff9a711bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:39.974647 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b7375-1c0e-4815-9c4b-87ff9a711bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:39.974917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b7375-1c0e-4815-9c4b-87ff9a711bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:39.975580 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b7375-1c0e-4815-9c4b-87ff9a711bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:39.977183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b7375-1c0e-4815-9c4b-87ff9a711bc1 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:39.978528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b2a3a9c4-49d7-45db-8ae9-d46fbc575a21 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:39.979793 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b2a3a9c4-49d7-45db-8ae9-d46fbc575a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:39.988398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1278/2585] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:39.988686 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1308/2586] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:39.992220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b7375-1c0e-4815-9c4b-87ff9a711bc1 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:39.992547 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3a9c4-49d7-45db-8ae9-d46fbc575a21 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.002509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b7375-1c0e-4815-9c4b-87ff9a711bc1 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.006390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3a9c4-49d7-45db-8ae9-d46fbc575a21 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.007389 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65e9e8-5605-452a-9aff-91979555a418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.008044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65e9e8-5605-452a-9aff-91979555a418 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.008324 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65e9e8-5605-452a-9aff-91979555a418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.008598 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65e9e8-5605-452a-9aff-91979555a418 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.009233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65e9e8-5605-452a-9aff-91979555a418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.010806 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65e9e8-5605-452a-9aff-91979555a418 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.011875 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69f316-a5c0-4b23-b64b-ef130e569ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.012477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69f316-a5c0-4b23-b64b-ef130e569ade admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.012758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69f316-a5c0-4b23-b64b-ef130e569ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.013064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69f316-a5c0-4b23-b64b-ef130e569ade admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.013851 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69f316-a5c0-4b23-b64b-ef130e569ade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:39:40.014177 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69f316-a5c0-4b23-b64b-ef130e569ade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.015805 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69f316-a5c0-4b23-b64b-ef130e569ade admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.021357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1279/2587] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:40.025368 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65e9e8-5605-452a-9aff-91979555a418 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.028055 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1309/2588] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 12:39:40.032688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69f316-a5c0-4b23-b64b-ef130e569ade admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.035766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65e9e8-5605-452a-9aff-91979555a418 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.040746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7fb9a-6a37-4a11-bb2b-69d1ef899bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.041440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7fb9a-6a37-4a11-bb2b-69d1ef899bb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.041723 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7fb9a-6a37-4a11-bb2b-69d1ef899bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.041999 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7fb9a-6a37-4a11-bb2b-69d1ef899bb0 admin admin] PATH_INFO: `/v3/projects/4729ba4b1330436cb7f83b80c35e6113/users/9622b449539945a0bf49c11d424787ad/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.045785 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69f316-a5c0-4b23-b64b-ef130e569ade admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.050523 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1a12f-9118-4bb5-b16f-552dcb6be573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.051118 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1a12f-9118-4bb5-b16f-552dcb6be573 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.051402 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1a12f-9118-4bb5-b16f-552dcb6be573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.051702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd1a12f-9118-4bb5-b16f-552dcb6be573 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.052326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1a12f-9118-4bb5-b16f-552dcb6be573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.054233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd1a12f-9118-4bb5-b16f-552dcb6be573 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.057777 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7fb9a-6a37-4a11-bb2b-69d1ef899bb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4729ba4b1330436cb7f83b80c35e6113, user_id=9622b449539945a0bf49c11d424787ad, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.060575 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7fb9a-6a37-4a11-bb2b-69d1ef899bb0 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.073371 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1280/2589] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:40 2022] PUT /identity/v3/projects/4729ba4b1330436cb7f83b80c35e6113/users/9622b449539945a0bf49c11d424787ad/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:40.082661 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1310/2590] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:40 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:39:40.085461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adae96-1caa-452d-b131-5ee1abc5c280 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.085796 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adae96-1caa-452d-b131-5ee1abc5c280 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.086140 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8adae96-1caa-452d-b131-5ee1abc5c280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.086838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1a12f-9118-4bb5-b16f-552dcb6be573 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.114634 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-b8adae96-1caa-452d-b131-5ee1abc5c280 None None] MFA Rules not processed for user `9622b449539945a0bf49c11d424787ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:40.115858 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b8adae96-1caa-452d-b131-5ee1abc5c280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:40.146119 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:40.148148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:40.152685 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd1a12f-9118-4bb5-b16f-552dcb6be573 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.158526 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df901a5-f9f9-458e-b758-faee195a74dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.159327 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df901a5-f9f9-458e-b758-faee195a74dd admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.159714 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df901a5-f9f9-458e-b758-faee195a74dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.160002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df901a5-f9f9-458e-b758-faee195a74dd admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.160640 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df901a5-f9f9-458e-b758-faee195a74dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.162767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df901a5-f9f9-458e-b758-faee195a74dd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.184432 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1281/2591] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:39:40.184628 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1311/2592] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:40 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:40.188620 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8adae96-1caa-452d-b131-5ee1abc5c280 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.188766 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df901a5-f9f9-458e-b758-faee195a74dd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.199323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df901a5-f9f9-458e-b758-faee195a74dd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.199492 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8adae96-1caa-452d-b131-5ee1abc5c280 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.204358 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556fef7-0f25-432e-9b69-7df293900bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.204752 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8aad1-d66d-41eb-90a5-71f4991aeeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.204949 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556fef7-0f25-432e-9b69-7df293900bab admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.205233 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556fef7-0f25-432e-9b69-7df293900bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.205343 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8aad1-d66d-41eb-90a5-71f4991aeeff admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.205511 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556fef7-0f25-432e-9b69-7df293900bab admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.205615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8aad1-d66d-41eb-90a5-71f4991aeeff admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.205870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d8aad1-d66d-41eb-90a5-71f4991aeeff admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.206143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556fef7-0f25-432e-9b69-7df293900bab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.206506 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8aad1-d66d-41eb-90a5-71f4991aeeff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.208202 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d8aad1-d66d-41eb-90a5-71f4991aeeff admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.208325 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556fef7-0f25-432e-9b69-7df293900bab admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.219220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1282/2593] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:40.223302 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8aad1-d66d-41eb-90a5-71f4991aeeff admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.235174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d8aad1-d66d-41eb-90a5-71f4991aeeff admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.238813 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1312/2594] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:40 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:39:40.239958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29487b14-1f2f-4b1f-970a-5ce7dc84e3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.240630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29487b14-1f2f-4b1f-970a-5ce7dc84e3dd admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.241015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29487b14-1f2f-4b1f-970a-5ce7dc84e3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.241376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29487b14-1f2f-4b1f-970a-5ce7dc84e3dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.242092 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29487b14-1f2f-4b1f-970a-5ce7dc84e3dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.242678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556fef7-0f25-432e-9b69-7df293900bab admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.243851 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29487b14-1f2f-4b1f-970a-5ce7dc84e3dd admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.255653 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1283/2595] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:39:40.258198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29487b14-1f2f-4b1f-970a-5ce7dc84e3dd admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.269125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29487b14-1f2f-4b1f-970a-5ce7dc84e3dd admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.312133 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556fef7-0f25-432e-9b69-7df293900bab admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.317671 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb20d7-c9ae-4d2c-ac50-625ded4744ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.318286 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb20d7-c9ae-4d2c-ac50-625ded4744ca admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.318578 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb20d7-c9ae-4d2c-ac50-625ded4744ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.318869 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdb20d7-c9ae-4d2c-ac50-625ded4744ca admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.319480 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb20d7-c9ae-4d2c-ac50-625ded4744ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.321959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdb20d7-c9ae-4d2c-ac50-625ded4744ca admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.328295 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfc463-7d1b-4eea-b268-6f23283e860a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.328955 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfc463-7d1b-4eea-b268-6f23283e860a admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.329238 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfc463-7d1b-4eea-b268-6f23283e860a admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.329515 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfc463-7d1b-4eea-b268-6f23283e860a admin admin] PATH_INFO: `/v3/projects/b0b2dfafc35a4e8ca969d5bfed36400e/users/e95eb284174b4fbdb237954f8b126aca/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.350972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfc463-7d1b-4eea-b268-6f23283e860a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0b2dfafc35a4e8ca969d5bfed36400e, user_id=e95eb284174b4fbdb237954f8b126aca, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.353997 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1313/2596] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:40 2022] POST /identity/v3/users => generated 303 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:39:40.354321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfc463-7d1b-4eea-b268-6f23283e860a admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.358263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb20d7-c9ae-4d2c-ac50-625ded4744ca admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.370406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdb20d7-c9ae-4d2c-ac50-625ded4744ca admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.375321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1284/2597] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:40 2022] PUT /identity/v3/projects/b0b2dfafc35a4e8ca969d5bfed36400e/users/e95eb284174b4fbdb237954f8b126aca/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:40.385639 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b187a-f7f5-4a01-bc74-66466223c9b3 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.385924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b187a-f7f5-4a01-bc74-66466223c9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.386197 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b187a-f7f5-4a01-bc74-66466223c9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.414156 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-039b187a-f7f5-4a01-bc74-66466223c9b3 None None] MFA Rules not processed for user `e95eb284174b4fbdb237954f8b126aca`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:40.415556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-039b187a-f7f5-4a01-bc74-66466223c9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:40.437372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ca60c-ed52-4cee-8bb1-0a61de847619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.437988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ca60c-ed52-4cee-8bb1-0a61de847619 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.438270 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ca60c-ed52-4cee-8bb1-0a61de847619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.438546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ca60c-ed52-4cee-8bb1-0a61de847619 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.439197 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ca60c-ed52-4cee-8bb1-0a61de847619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.440837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ca60c-ed52-4cee-8bb1-0a61de847619 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.455067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1314/2598] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:39:40.456146 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ca60c-ed52-4cee-8bb1-0a61de847619 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.471062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ca60c-ed52-4cee-8bb1-0a61de847619 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.476111 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4758db4-72da-4a13-83c3-4f9c61f69a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.476771 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4758db4-72da-4a13-83c3-4f9c61f69a9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.477081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4758db4-72da-4a13-83c3-4f9c61f69a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.477359 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4758db4-72da-4a13-83c3-4f9c61f69a9f admin admin] PATH_INFO: `/v3/projects/9091e79697414317bd2b3b75eecc6159/users/84607cc4055a4a61b32ca641c3c46318/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.485238 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1285/2599] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:39:40.489734 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b187a-f7f5-4a01-bc74-66466223c9b3 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.492984 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4758db4-72da-4a13-83c3-4f9c61f69a9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9091e79697414317bd2b3b75eecc6159, user_id=84607cc4055a4a61b32ca641c3c46318, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.495720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4758db4-72da-4a13-83c3-4f9c61f69a9f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.500091 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b187a-f7f5-4a01-bc74-66466223c9b3 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.505291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7176174-772d-4bc5-94d9-df00668cdc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.505880 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7176174-772d-4bc5-94d9-df00668cdc5d admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.506159 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7176174-772d-4bc5-94d9-df00668cdc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.506429 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7176174-772d-4bc5-94d9-df00668cdc5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.507045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7176174-772d-4bc5-94d9-df00668cdc5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.508949 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7176174-772d-4bc5-94d9-df00668cdc5d admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.509058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1315/2600] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:40 2022] PUT /identity/v3/projects/9091e79697414317bd2b3b75eecc6159/users/84607cc4055a4a61b32ca641c3c46318/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:40.512782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4758db4-72da-4a13-83c3-4f9c61f69a9f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.537909 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1286/2601] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:40 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:39:40.541894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7176174-772d-4bc5-94d9-df00668cdc5d admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.542083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:40.550899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:40.583062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4758db4-72da-4a13-83c3-4f9c61f69a9f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.610722 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7176174-772d-4bc5-94d9-df00668cdc5d admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.615337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5902352-ae64-4a2e-8d3b-ddb48c77e1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.615972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5902352-ae64-4a2e-8d3b-ddb48c77e1b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.616254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5902352-ae64-4a2e-8d3b-ddb48c77e1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.616529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5902352-ae64-4a2e-8d3b-ddb48c77e1b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.617154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5902352-ae64-4a2e-8d3b-ddb48c77e1b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.619071 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5902352-ae64-4a2e-8d3b-ddb48c77e1b4 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.641880 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1287/2602] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:40 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:39:40.644467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9771a-0439-4c4d-a613-51cd1f75c6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.645196 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9771a-0439-4c4d-a613-51cd1f75c6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.645484 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9771a-0439-4c4d-a613-51cd1f75c6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.645766 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9771a-0439-4c4d-a613-51cd1f75c6d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.645894 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5902352-ae64-4a2e-8d3b-ddb48c77e1b4 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.646412 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9771a-0439-4c4d-a613-51cd1f75c6d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.648056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9771a-0439-4c4d-a613-51cd1f75c6d9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.656153 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5902352-ae64-4a2e-8d3b-ddb48c77e1b4 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.660815 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04fcad-3e0d-45d2-bad6-334488bc82cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.661282 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1316/2603] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:39:40.661405 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04fcad-3e0d-45d2-bad6-334488bc82cc admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.661677 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04fcad-3e0d-45d2-bad6-334488bc82cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.661978 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04fcad-3e0d-45d2-bad6-334488bc82cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.662680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04fcad-3e0d-45d2-bad6-334488bc82cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.664314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04fcad-3e0d-45d2-bad6-334488bc82cc admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.666208 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9771a-0439-4c4d-a613-51cd1f75c6d9 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.678627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1288/2604] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:39:40.679947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04fcad-3e0d-45d2-bad6-334488bc82cc admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.681375 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9771a-0439-4c4d-a613-51cd1f75c6d9 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.686093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005003c-e7b1-413d-b2d1-771efbe6f30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.686745 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005003c-e7b1-413d-b2d1-771efbe6f30b admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.687027 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005003c-e7b1-413d-b2d1-771efbe6f30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.687311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005003c-e7b1-413d-b2d1-771efbe6f30b admin admin] PATH_INFO: `/v3/projects/9091e79697414317bd2b3b75eecc6159/users/84607cc4055a4a61b32ca641c3c46318/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.689268 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005003c-e7b1-413d-b2d1-771efbe6f30b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9091e79697414317bd2b3b75eecc6159, user_id=84607cc4055a4a61b32ca641c3c46318, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.689538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04fcad-3e0d-45d2-bad6-334488bc82cc admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.691962 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005003c-e7b1-413d-b2d1-771efbe6f30b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.694030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508d34d-60a3-4a35-bd5e-09ef7ccdf254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.694658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508d34d-60a3-4a35-bd5e-09ef7ccdf254 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.694930 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508d34d-60a3-4a35-bd5e-09ef7ccdf254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.695203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508d34d-60a3-4a35-bd5e-09ef7ccdf254 admin admin] PATH_INFO: `/v3/projects/85bcae21f17645c4ad38981dcd8218ec/users/617d18a097e1411e867ff2d582beb2d7/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.703316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1317/2605] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:40 2022] PUT /identity/v3/projects/9091e79697414317bd2b3b75eecc6159/users/84607cc4055a4a61b32ca641c3c46318/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:40.712270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508d34d-60a3-4a35-bd5e-09ef7ccdf254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85bcae21f17645c4ad38981dcd8218ec, user_id=617d18a097e1411e867ff2d582beb2d7, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.714102 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553debf-c160-461d-9f9e-46f64b5399ed None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.714335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553debf-c160-461d-9f9e-46f64b5399ed None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.714616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553debf-c160-461d-9f9e-46f64b5399ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.714975 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508d34d-60a3-4a35-bd5e-09ef7ccdf254 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.729173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1289/2606] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:40 2022] PUT /identity/v3/projects/85bcae21f17645c4ad38981dcd8218ec/users/617d18a097e1411e867ff2d582beb2d7/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:40.733764 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508d34d-60a3-4a35-bd5e-09ef7ccdf254 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.747616 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-8553debf-c160-461d-9f9e-46f64b5399ed None None] MFA Rules not processed for user `84607cc4055a4a61b32ca641c3c46318`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:40.748855 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8553debf-c160-461d-9f9e-46f64b5399ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:40.800297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508d34d-60a3-4a35-bd5e-09ef7ccdf254 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.805956 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8b0fc-0799-4dc8-9845-2a590345de26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.806561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8b0fc-0799-4dc8-9845-2a590345de26 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.806839 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8b0fc-0799-4dc8-9845-2a590345de26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.807109 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8b0fc-0799-4dc8-9845-2a590345de26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.807769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8b0fc-0799-4dc8-9845-2a590345de26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.809374 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8b0fc-0799-4dc8-9845-2a590345de26 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.819885 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1290/2607] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:40 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:40.822814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1318/2608] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:40 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:40.823689 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8b0fc-0799-4dc8-9845-2a590345de26 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.834752 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8b0fc-0799-4dc8-9845-2a590345de26 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.835174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553debf-c160-461d-9f9e-46f64b5399ed None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:40.837017 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8553debf-c160-461d-9f9e-46f64b5399ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:40.839835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5153e-cc59-497f-b397-f305c1f5622b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.840582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5153e-cc59-497f-b397-f305c1f5622b admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.840977 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5153e-cc59-497f-b397-f305c1f5622b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.841868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5153e-cc59-497f-b397-f305c1f5622b admin admin] PATH_INFO: `/v3/projects/85bcae21f17645c4ad38981dcd8218ec/users/617d18a097e1411e867ff2d582beb2d7/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.844410 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5153e-cc59-497f-b397-f305c1f5622b admin admin] RBAC: Authorizing `identity:create_grant(project_id=85bcae21f17645c4ad38981dcd8218ec, user_id=617d18a097e1411e867ff2d582beb2d7, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:40.847917 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5153e-cc59-497f-b397-f305c1f5622b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:40.861371 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1291/2609] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:40 2022] PUT /identity/v3/projects/85bcae21f17645c4ad38981dcd8218ec/users/617d18a097e1411e867ff2d582beb2d7/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:40.873061 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0c5f5-9300-4a28-bf88-94d39eacbb5f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.873577 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0c5f5-9300-4a28-bf88-94d39eacbb5f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.874369 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0c5f5-9300-4a28-bf88-94d39eacbb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.896156 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:40.912220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-47d0c5f5-9300-4a28-bf88-94d39eacbb5f None None] MFA Rules not processed for user `617d18a097e1411e867ff2d582beb2d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:40.913499 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-47d0c5f5-9300-4a28-bf88-94d39eacbb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:40.916313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553debf-c160-461d-9f9e-46f64b5399ed None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:40.951538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:40.973981 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28244c8a-4b38-4b7e-b0e9-29ed98daaf4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:40.974657 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28244c8a-4b38-4b7e-b0e9-29ed98daaf4c service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:40.975050 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28244c8a-4b38-4b7e-b0e9-29ed98daaf4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:40.975524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28244c8a-4b38-4b7e-b0e9-29ed98daaf4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:40.995498 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1292/2610] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:39:41.009314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0c5f5-9300-4a28-bf88-94d39eacbb5f None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:41.011065 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-47d0c5f5-9300-4a28-bf88-94d39eacbb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:41.023385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0c5f5-9300-4a28-bf88-94d39eacbb5f None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:41.027453 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801117cc-ccf2-48b5-bd2c-b67a33426229 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:41.028201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801117cc-ccf2-48b5-bd2c-b67a33426229 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:41.028486 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801117cc-ccf2-48b5-bd2c-b67a33426229 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:41.029453 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801117cc-ccf2-48b5-bd2c-b67a33426229 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:41.039130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801117cc-ccf2-48b5-bd2c-b67a33426229 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:41.040718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801117cc-ccf2-48b5-bd2c-b67a33426229 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:41.043659 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28244c8a-4b38-4b7e-b0e9-29ed98daaf4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:41.045629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28244c8a-4b38-4b7e-b0e9-29ed98daaf4c service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:41.051116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1293/2611] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:39:41.055905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1319/2612] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:40 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:41.097727 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801117cc-ccf2-48b5-bd2c-b67a33426229 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:41.104631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28244c8a-4b38-4b7e-b0e9-29ed98daaf4c service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:41.106487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801117cc-ccf2-48b5-bd2c-b67a33426229 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:41.110802 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5febd-ed9f-4dbc-b8dd-691f6eafdaa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:41.111505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5febd-ed9f-4dbc-b8dd-691f6eafdaa9 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:41.111874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5febd-ed9f-4dbc-b8dd-691f6eafdaa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:41.112211 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5febd-ed9f-4dbc-b8dd-691f6eafdaa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:41.113891 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28244c8a-4b38-4b7e-b0e9-29ed98daaf4c service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:41.117573 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b02ba-6061-4d52-be95-b628b3a30833 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:41.118173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b02ba-6061-4d52-be95-b628b3a30833 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:41.118453 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b02ba-6061-4d52-be95-b628b3a30833 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:41.118720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b02ba-6061-4d52-be95-b628b3a30833 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:41.182805 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5febd-ed9f-4dbc-b8dd-691f6eafdaa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:41.183700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b02ba-6061-4d52-be95-b628b3a30833 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:41.184287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5febd-ed9f-4dbc-b8dd-691f6eafdaa9 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:41.185829 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b02ba-6061-4d52-be95-b628b3a30833 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:41.193915 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1294/2613] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:41.194831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1320/2614] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:39:44.760026 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5febd-ed9f-4dbc-b8dd-691f6eafdaa9 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:44.822406 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5febd-ed9f-4dbc-b8dd-691f6eafdaa9 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:44.827687 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757532c-3b7f-4798-bc56-687c5a7b960e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:44.828333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757532c-3b7f-4798-bc56-687c5a7b960e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:44.828608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757532c-3b7f-4798-bc56-687c5a7b960e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:44.828875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757532c-3b7f-4798-bc56-687c5a7b960e admin admin] PATH_INFO: `/v3/users/e95eb284174b4fbdb237954f8b126aca` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:44.829966 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4757532c-3b7f-4798-bc56-687c5a7b960e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e95eb284174b4fbdb237954f8b126aca)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:44.831891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4757532c-3b7f-4798-bc56-687c5a7b960e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:44.879394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-4757532c-3b7f-4798-bc56-687c5a7b960e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e95eb284174b4fbdb237954f8b126aca'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:44.880188 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-4757532c-3b7f-4798-bc56-687c5a7b960e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e95eb284174b4fbdb237954f8b126aca'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:44.880870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-4757532c-3b7f-4798-bc56-687c5a7b960e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e95eb284174b4fbdb237954f8b126aca'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:44.886530 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:44.889258 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-4757532c-3b7f-4798-bc56-687c5a7b960e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e95eb284174b4fbdb237954f8b126aca'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:44.897293 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-4757532c-3b7f-4798-bc56-687c5a7b960e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e95eb284174b4fbdb237954f8b126aca'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:44.915066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1295/2615] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:44 2022] DELETE /identity/v3/users/e95eb284174b4fbdb237954f8b126aca => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:44.919605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08de9f0-cc25-49ba-89c1-1c56f00e5484 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:44.920141 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08de9f0-cc25-49ba-89c1-1c56f00e5484 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:44.920295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08de9f0-cc25-49ba-89c1-1c56f00e5484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:44.938897 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b08de9f0-cc25-49ba-89c1-1c56f00e5484 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:44.940129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b08de9f0-cc25-49ba-89c1-1c56f00e5484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:44.981105 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:45.000026 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1321/2616] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:45.009888 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757532c-3b7f-4798-bc56-687c5a7b960e admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:45.011372 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-4757532c-3b7f-4798-bc56-687c5a7b960e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:45.070067 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757532c-3b7f-4798-bc56-687c5a7b960e admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:45.073697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00367d84-3c34-4b1e-ad8e-56be59c41b96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:45.074229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00367d84-3c34-4b1e-ad8e-56be59c41b96 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:45.074755 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00367d84-3c34-4b1e-ad8e-56be59c41b96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:45.075030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00367d84-3c34-4b1e-ad8e-56be59c41b96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:45.084233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00367d84-3c34-4b1e-ad8e-56be59c41b96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:45.085655 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00367d84-3c34-4b1e-ad8e-56be59c41b96 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:45.095489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1296/2617] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:45.377112 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08de9f0-cc25-49ba-89c1-1c56f00e5484 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:45.378511 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b08de9f0-cc25-49ba-89c1-1c56f00e5484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:45.393940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08de9f0-cc25-49ba-89c1-1c56f00e5484 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:45.399058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b152d47-08a1-4d66-a587-93ff883dc3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:45.399525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b152d47-08a1-4d66-a587-93ff883dc3b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:45.399855 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b152d47-08a1-4d66-a587-93ff883dc3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:45.400138 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b152d47-08a1-4d66-a587-93ff883dc3b7 admin admin] PATH_INFO: `/v3/projects/b0b2dfafc35a4e8ca969d5bfed36400e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:45.401276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b152d47-08a1-4d66-a587-93ff883dc3b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0b2dfafc35a4e8ca969d5bfed36400e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:45.403334 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b152d47-08a1-4d66-a587-93ff883dc3b7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:45.451247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1b152d47-08a1-4d66-a587-93ff883dc3b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0b2dfafc35a4e8ca969d5bfed36400e', 'request_id': 'req-1b152d47-08a1-4d66-a587-93ff883dc3b7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:45.456765 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1b152d47-08a1-4d66-a587-93ff883dc3b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0b2dfafc35a4e8ca969d5bfed36400e', 'request_id': 'req-1b152d47-08a1-4d66-a587-93ff883dc3b7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:45.464050 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1b152d47-08a1-4d66-a587-93ff883dc3b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0b2dfafc35a4e8ca969d5bfed36400e', 'request_id': 'req-1b152d47-08a1-4d66-a587-93ff883dc3b7'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:45.475470 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1b152d47-08a1-4d66-a587-93ff883dc3b7 admin admin] The token cache is being invalidate because project b0b2dfafc35a4e8ca969d5bfed36400e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:45.475795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1b152d47-08a1-4d66-a587-93ff883dc3b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:45.476416 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1b152d47-08a1-4d66-a587-93ff883dc3b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:45.477909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1322/2618] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:45 2022] DELETE /identity/v3/projects/b0b2dfafc35a4e8ca969d5bfed36400e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:45.482872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00367d84-3c34-4b1e-ad8e-56be59c41b96 service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:45.492474 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00367d84-3c34-4b1e-ad8e-56be59c41b96 service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:45.547329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90b79b-46e6-49c3-a0a3-166156946938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:45.548005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90b79b-46e6-49c3-a0a3-166156946938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:45.548286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90b79b-46e6-49c3-a0a3-166156946938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:45.548560 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90b79b-46e6-49c3-a0a3-166156946938 admin admin] PATH_INFO: `/v3/users/617d18a097e1411e867ff2d582beb2d7` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:45.549677 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90b79b-46e6-49c3-a0a3-166156946938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=617d18a097e1411e867ff2d582beb2d7)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:45.551580 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90b79b-46e6-49c3-a0a3-166156946938 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:45.598055 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2e90b79b-46e6-49c3-a0a3-166156946938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '617d18a097e1411e867ff2d582beb2d7'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:45.598627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2e90b79b-46e6-49c3-a0a3-166156946938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '617d18a097e1411e867ff2d582beb2d7'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:45.599229 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2e90b79b-46e6-49c3-a0a3-166156946938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '617d18a097e1411e867ff2d582beb2d7'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:45.606261 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2e90b79b-46e6-49c3-a0a3-166156946938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '617d18a097e1411e867ff2d582beb2d7'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:45.613605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2e90b79b-46e6-49c3-a0a3-166156946938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '617d18a097e1411e867ff2d582beb2d7'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:45.623696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:45.632893 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1297/2619] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:45 2022] DELETE /identity/v3/users/617d18a097e1411e867ff2d582beb2d7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:45.908390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b152d47-08a1-4d66-a587-93ff883dc3b7 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:45.909951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1b152d47-08a1-4d66-a587-93ff883dc3b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:45.923598 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b152d47-08a1-4d66-a587-93ff883dc3b7 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:45.982035 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf3f67c-bc33-4e27-9b7e-e565cb6aaa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:45.982732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3f67c-bc33-4e27-9b7e-e565cb6aaa4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:45.983017 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3f67c-bc33-4e27-9b7e-e565cb6aaa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:45.983292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3f67c-bc33-4e27-9b7e-e565cb6aaa4e admin admin] PATH_INFO: `/v3/projects/85bcae21f17645c4ad38981dcd8218ec` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:45.984445 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3f67c-bc33-4e27-9b7e-e565cb6aaa4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=85bcae21f17645c4ad38981dcd8218ec)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:45.986353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf3f67c-bc33-4e27-9b7e-e565cb6aaa4e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.032359 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6bf3f67c-bc33-4e27-9b7e-e565cb6aaa4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85bcae21f17645c4ad38981dcd8218ec', 'request_id': 'req-6bf3f67c-bc33-4e27-9b7e-e565cb6aaa4e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:46.037884 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6bf3f67c-bc33-4e27-9b7e-e565cb6aaa4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85bcae21f17645c4ad38981dcd8218ec', 'request_id': 'req-6bf3f67c-bc33-4e27-9b7e-e565cb6aaa4e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:46.045302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6bf3f67c-bc33-4e27-9b7e-e565cb6aaa4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85bcae21f17645c4ad38981dcd8218ec', 'request_id': 'req-6bf3f67c-bc33-4e27-9b7e-e565cb6aaa4e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:46.052344 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:46.057473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6bf3f67c-bc33-4e27-9b7e-e565cb6aaa4e admin admin] The token cache is being invalidate because project 85bcae21f17645c4ad38981dcd8218ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:46.058104 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6bf3f67c-bc33-4e27-9b7e-e565cb6aaa4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:46.059189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-6bf3f67c-bc33-4e27-9b7e-e565cb6aaa4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:46.061376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1323/2620] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:45 2022] DELETE /identity/v3/projects/85bcae21f17645c4ad38981dcd8218ec => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:46.066381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b79e5-c012-45ca-9979-2759a31b2a09 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.066869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b79e5-c012-45ca-9979-2759a31b2a09 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.066965 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b79e5-c012-45ca-9979-2759a31b2a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.089186 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-931b79e5-c012-45ca-9979-2759a31b2a09 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:46.090398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-931b79e5-c012-45ca-9979-2759a31b2a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:46.149117 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1298/2621] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:46.158424 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad48ae-049f-40be-aaf7-a917b6dc396f None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.158712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad48ae-049f-40be-aaf7-a917b6dc396f None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.159134 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad48ae-049f-40be-aaf7-a917b6dc396f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.179130 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-36ad48ae-049f-40be-aaf7-a917b6dc396f None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:46.180372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-36ad48ae-049f-40be-aaf7-a917b6dc396f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:46.188329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1324/2622] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:46.192463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b79e5-c012-45ca-9979-2759a31b2a09 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.202468 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b79e5-c012-45ca-9979-2759a31b2a09 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.207004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bd1c0-6890-4f72-b5cf-231ce111577d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.207598 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bd1c0-6890-4f72-b5cf-231ce111577d admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.207916 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bd1c0-6890-4f72-b5cf-231ce111577d admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.208193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bd1c0-6890-4f72-b5cf-231ce111577d admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.208835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10bd1c0-6890-4f72-b5cf-231ce111577d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:39:46.209141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10bd1c0-6890-4f72-b5cf-231ce111577d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:46.210732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10bd1c0-6890-4f72-b5cf-231ce111577d admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.222599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1299/2623] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:39:46.226835 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad48ae-049f-40be-aaf7-a917b6dc396f None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.236710 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad48ae-049f-40be-aaf7-a917b6dc396f None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.241369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e483b0e-df55-4aab-bb9c-261b0cb93e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.241956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e483b0e-df55-4aab-bb9c-261b0cb93e3f admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.242232 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e483b0e-df55-4aab-bb9c-261b0cb93e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.242501 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e483b0e-df55-4aab-bb9c-261b0cb93e3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.243112 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e483b0e-df55-4aab-bb9c-261b0cb93e3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:46.245054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e483b0e-df55-4aab-bb9c-261b0cb93e3f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.272689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1325/2624] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:46 2022] POST /identity/v3/projects => generated 409 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:46.276936 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bd1c0-6890-4f72-b5cf-231ce111577d admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.287441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bd1c0-6890-4f72-b5cf-231ce111577d admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.343208 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f42e0-3a9f-4edf-934e-e326cf97cba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.343904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f42e0-3a9f-4edf-934e-e326cf97cba9 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.344187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f42e0-3a9f-4edf-934e-e326cf97cba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.344466 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f42e0-3a9f-4edf-934e-e326cf97cba9 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.345087 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f42e0-3a9f-4edf-934e-e326cf97cba9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:46.346993 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f42e0-3a9f-4edf-934e-e326cf97cba9 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.374081 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1300/2625] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:46 2022] POST /identity/v3/users => generated 305 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:39:46.375981 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e483b0e-df55-4aab-bb9c-261b0cb93e3f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.387251 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e483b0e-df55-4aab-bb9c-261b0cb93e3f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.392563 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b7e0d-b015-4abb-87be-ba7af00fb764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.393172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b7e0d-b015-4abb-87be-ba7af00fb764 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.393438 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b7e0d-b015-4abb-87be-ba7af00fb764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.393704 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b7e0d-b015-4abb-87be-ba7af00fb764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.394497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b7e0d-b015-4abb-87be-ba7af00fb764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:46.396231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b7e0d-b015-4abb-87be-ba7af00fb764 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.412231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1326/2626] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:46.413393 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f42e0-3a9f-4edf-934e-e326cf97cba9 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.424030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f42e0-3a9f-4edf-934e-e326cf97cba9 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.429140 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754a749-281e-4ddd-9e36-141d76b72456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.431192 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754a749-281e-4ddd-9e36-141d76b72456 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.431406 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754a749-281e-4ddd-9e36-141d76b72456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.431611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754a749-281e-4ddd-9e36-141d76b72456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.432371 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754a749-281e-4ddd-9e36-141d76b72456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:46.433900 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754a749-281e-4ddd-9e36-141d76b72456 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.448723 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1301/2627] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:39:46.453747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b7e0d-b015-4abb-87be-ba7af00fb764 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.464535 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b7e0d-b015-4abb-87be-ba7af00fb764 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.470163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf01a8-5b8d-4642-ac23-faa7c47e8852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.470827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf01a8-5b8d-4642-ac23-faa7c47e8852 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.471110 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf01a8-5b8d-4642-ac23-faa7c47e8852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.471384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf01a8-5b8d-4642-ac23-faa7c47e8852 admin admin] PATH_INFO: `/v3/projects/8b15f2e9825749f1af329d4341894f93/users/9a2d5eb11fd541ddb0b2817312c3acaf/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.488248 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf01a8-5b8d-4642-ac23-faa7c47e8852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b15f2e9825749f1af329d4341894f93, user_id=9a2d5eb11fd541ddb0b2817312c3acaf, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:46.490951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf01a8-5b8d-4642-ac23-faa7c47e8852 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.504806 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1327/2628] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:46 2022] PUT /identity/v3/projects/8b15f2e9825749f1af329d4341894f93/users/9a2d5eb11fd541ddb0b2817312c3acaf/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:46.516187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087d752-d9c6-4d0e-9ee2-f6cf21f4818f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.517019 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087d752-d9c6-4d0e-9ee2-f6cf21f4818f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.517301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087d752-d9c6-4d0e-9ee2-f6cf21f4818f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.549004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-b087d752-d9c6-4d0e-9ee2-f6cf21f4818f None None] MFA Rules not processed for user `9a2d5eb11fd541ddb0b2817312c3acaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:46.550234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b087d752-d9c6-4d0e-9ee2-f6cf21f4818f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:46.598606 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:46.618472 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1302/2629] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:46 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:46.623560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf01a8-5b8d-4642-ac23-faa7c47e8852 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.686546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf01a8-5b8d-4642-ac23-faa7c47e8852 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.691285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996a645-9c4a-4710-92ae-abcbe2f5d8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.691918 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996a645-9c4a-4710-92ae-abcbe2f5d8bc admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.692202 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996a645-9c4a-4710-92ae-abcbe2f5d8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.692476 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996a645-9c4a-4710-92ae-abcbe2f5d8bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.693098 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996a645-9c4a-4710-92ae-abcbe2f5d8bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:46.695033 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996a645-9c4a-4710-92ae-abcbe2f5d8bc admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.726751 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1328/2630] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:46 2022] POST /identity/v3/projects => generated 407 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:39:46.728003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087d752-d9c6-4d0e-9ee2-f6cf21f4818f None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.740015 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087d752-d9c6-4d0e-9ee2-f6cf21f4818f None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.801905 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ffd8e-193f-4ce9-8ebd-0a438412dcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.802515 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ffd8e-193f-4ce9-8ebd-0a438412dcf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.802792 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ffd8e-193f-4ce9-8ebd-0a438412dcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.803059 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ffd8e-193f-4ce9-8ebd-0a438412dcf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.803703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ffd8e-193f-4ce9-8ebd-0a438412dcf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:46.805609 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ffd8e-193f-4ce9-8ebd-0a438412dcf9 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.831394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1303/2631] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:46 2022] POST /identity/v3/users => generated 310 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:46.832657 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996a645-9c4a-4710-92ae-abcbe2f5d8bc admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.843354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996a645-9c4a-4710-92ae-abcbe2f5d8bc admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.849364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6362bc9-359c-4eed-bda2-8468c5857bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.849981 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6362bc9-359c-4eed-bda2-8468c5857bcb admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.850267 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6362bc9-359c-4eed-bda2-8468c5857bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.850544 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6362bc9-359c-4eed-bda2-8468c5857bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.851226 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6362bc9-359c-4eed-bda2-8468c5857bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:46.852867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6362bc9-359c-4eed-bda2-8468c5857bcb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.865473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1329/2632] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:39:46.869766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ffd8e-193f-4ce9-8ebd-0a438412dcf9 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.884980 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ffd8e-193f-4ce9-8ebd-0a438412dcf9 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:46.890808 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01024a3e-1d05-4dbb-91ed-a27f35bbc5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:46.891469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01024a3e-1d05-4dbb-91ed-a27f35bbc5c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:46.891914 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01024a3e-1d05-4dbb-91ed-a27f35bbc5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:46.892185 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01024a3e-1d05-4dbb-91ed-a27f35bbc5c4 admin admin] PATH_INFO: `/v3/projects/3095403f34394e3e8623d9055f54841e/users/3445d981e433449b8ca8988ac7e7ceba/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:46.912444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01024a3e-1d05-4dbb-91ed-a27f35bbc5c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3095403f34394e3e8623d9055f54841e, user_id=3445d981e433449b8ca8988ac7e7ceba, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:46.916223 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01024a3e-1d05-4dbb-91ed-a27f35bbc5c4 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:46.935804 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1304/2633] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:46 2022] PUT /identity/v3/projects/3095403f34394e3e8623d9055f54841e/users/3445d981e433449b8ca8988ac7e7ceba/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:46.937511 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6362bc9-359c-4eed-bda2-8468c5857bcb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:46.947649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6362bc9-359c-4eed-bda2-8468c5857bcb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:47.017419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296dbd52-8bb3-49ad-80d6-a92a2e79165f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:47.018252 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296dbd52-8bb3-49ad-80d6-a92a2e79165f admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:47.018648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296dbd52-8bb3-49ad-80d6-a92a2e79165f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:47.019082 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296dbd52-8bb3-49ad-80d6-a92a2e79165f admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:47.019757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296dbd52-8bb3-49ad-80d6-a92a2e79165f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:47.021382 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296dbd52-8bb3-49ad-80d6-a92a2e79165f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:47.032768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1330/2634] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:39:47.037444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01024a3e-1d05-4dbb-91ed-a27f35bbc5c4 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:47.050925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296dbd52-8bb3-49ad-80d6-a92a2e79165f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:47.051871 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01024a3e-1d05-4dbb-91ed-a27f35bbc5c4 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:47.053113 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-296dbd52-8bb3-49ad-80d6-a92a2e79165f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:47.057152 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1db44-2797-4756-8021-4d1f6a8227fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:47.057886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1db44-2797-4756-8021-4d1f6a8227fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:47.058232 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1db44-2797-4756-8021-4d1f6a8227fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:47.058606 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1db44-2797-4756-8021-4d1f6a8227fe admin admin] PATH_INFO: `/v3/projects/3095403f34394e3e8623d9055f54841e/users/3445d981e433449b8ca8988ac7e7ceba/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:47.061275 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1db44-2797-4756-8021-4d1f6a8227fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=3095403f34394e3e8623d9055f54841e, user_id=3445d981e433449b8ca8988ac7e7ceba, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:47.062265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:47.064396 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1db44-2797-4756-8021-4d1f6a8227fe admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:47.066369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296dbd52-8bb3-49ad-80d6-a92a2e79165f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:47.073412 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6cd243-b4cb-442f-b26b-2609ff709181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:47.074381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cd243-b4cb-442f-b26b-2609ff709181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:47.074381 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cd243-b4cb-442f-b26b-2609ff709181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:47.074977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cd243-b4cb-442f-b26b-2609ff709181 admin admin] PATH_INFO: `/v3/users/9622b449539945a0bf49c11d424787ad` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:47.076271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6cd243-b4cb-442f-b26b-2609ff709181 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9622b449539945a0bf49c11d424787ad)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:47.077319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1305/2635] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:47 2022] PUT /identity/v3/projects/3095403f34394e3e8623d9055f54841e/users/3445d981e433449b8ca8988ac7e7ceba/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:47.078345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6cd243-b4cb-442f-b26b-2609ff709181 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:47.088598 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b585856-03d8-41e9-86b9-d6aa5d2c6393 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:47.089703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b585856-03d8-41e9-86b9-d6aa5d2c6393 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:47.089703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b585856-03d8-41e9-86b9-d6aa5d2c6393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:47.120690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-6b585856-03d8-41e9-86b9-d6aa5d2c6393 None None] MFA Rules not processed for user `3445d981e433449b8ca8988ac7e7ceba`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:47.121619 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-6b585856-03d8-41e9-86b9-d6aa5d2c6393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:47.164389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8c6cd243-b4cb-442f-b26b-2609ff709181 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9622b449539945a0bf49c11d424787ad'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:47.165132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8c6cd243-b4cb-442f-b26b-2609ff709181 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9622b449539945a0bf49c11d424787ad'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:47.176452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8c6cd243-b4cb-442f-b26b-2609ff709181 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9622b449539945a0bf49c11d424787ad'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:47.188623 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8c6cd243-b4cb-442f-b26b-2609ff709181 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9622b449539945a0bf49c11d424787ad'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:47.205450 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8c6cd243-b4cb-442f-b26b-2609ff709181 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9622b449539945a0bf49c11d424787ad'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:47.213013 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1331/2636] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:47 2022] DELETE /identity/v3/users/9622b449539945a0bf49c11d424787ad => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:47.218425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5067f-f86f-4ddc-82f1-66d7518124a1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:47.218724 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5067f-f86f-4ddc-82f1-66d7518124a1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:47.218991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5067f-f86f-4ddc-82f1-66d7518124a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:47.228572 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1306/2637] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:47 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:39:47.240025 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b585856-03d8-41e9-86b9-d6aa5d2c6393 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:47.241741 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-6b585856-03d8-41e9-86b9-d6aa5d2c6393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:47.244874 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1ad5067f-f86f-4ddc-82f1-66d7518124a1 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:47.246062 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1ad5067f-f86f-4ddc-82f1-66d7518124a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:47.283116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:47.321612 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b585856-03d8-41e9-86b9-d6aa5d2c6393 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:47.326558 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1332/2638] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:47.336079 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5067f-f86f-4ddc-82f1-66d7518124a1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:47.337457 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1ad5067f-f86f-4ddc-82f1-66d7518124a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:47.373512 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0d612-841b-4744-9e04-554578bb20e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:47.374119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0d612-841b-4744-9e04-554578bb20e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:47.374516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0d612-841b-4744-9e04-554578bb20e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:47.374792 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0d612-841b-4744-9e04-554578bb20e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:47.376553 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f2b0d612-841b-4744-9e04-554578bb20e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:47.396246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5067f-f86f-4ddc-82f1-66d7518124a1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:47.400021 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34ad42-04f2-444c-9c34-7637c9f015ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:47.400629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34ad42-04f2-444c-9c34-7637c9f015ee service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:47.400911 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34ad42-04f2-444c-9c34-7637c9f015ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:47.401185 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34ad42-04f2-444c-9c34-7637c9f015ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:47.410562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34ad42-04f2-444c-9c34-7637c9f015ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:47.412052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34ad42-04f2-444c-9c34-7637c9f015ee service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:47.422951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1333/2639] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:39:47.440829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0d612-841b-4744-9e04-554578bb20e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:47.442237 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0d612-841b-4744-9e04-554578bb20e8 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:47.454295 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1307/2640] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:47 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:39:47.681700 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34ad42-04f2-444c-9c34-7637c9f015ee service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:47.682900 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8a34ad42-04f2-444c-9c34-7637c9f015ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:47.697611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34ad42-04f2-444c-9c34-7637c9f015ee service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:47.703660 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3185730-2ad3-4b36-bbb3-d8fe48669d25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:47.704247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3185730-2ad3-4b36-bbb3-d8fe48669d25 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:47.705331 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3185730-2ad3-4b36-bbb3-d8fe48669d25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:47.705331 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3185730-2ad3-4b36-bbb3-d8fe48669d25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:47.707030 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c3185730-2ad3-4b36-bbb3-d8fe48669d25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:47.729393 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:47.805995 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3185730-2ad3-4b36-bbb3-d8fe48669d25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:47.807460 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3185730-2ad3-4b36-bbb3-d8fe48669d25 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:47.811821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0d612-841b-4744-9e04-554578bb20e8 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:47.813260 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f2b0d612-841b-4744-9e04-554578bb20e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:47.820967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1334/2641] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:47 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:47.826993 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0d612-841b-4744-9e04-554578bb20e8 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:47.831829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:47.832672 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:47.833049 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:47.833391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8 admin admin] PATH_INFO: `/v3/projects/4729ba4b1330436cb7f83b80c35e6113` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:47.834630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4729ba4b1330436cb7f83b80c35e6113)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:47.836845 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:47.891816 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4729ba4b1330436cb7f83b80c35e6113', 'request_id': 'req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:47.897968 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4729ba4b1330436cb7f83b80c35e6113', 'request_id': 'req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:47.906743 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4729ba4b1330436cb7f83b80c35e6113', 'request_id': 'req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:47.918538 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8 admin admin] The token cache is being invalidate because project 4729ba4b1330436cb7f83b80c35e6113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:47.918817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:47.920531 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:47.922082 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1308/2642] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:47 2022] DELETE /identity/v3/projects/4729ba4b1330436cb7f83b80c35e6113 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:39:47.926097 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3185730-2ad3-4b36-bbb3-d8fe48669d25 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:47.938250 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3185730-2ad3-4b36-bbb3-d8fe48669d25 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:48.002091 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0d053-7d0f-466b-9e70-492273261b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:48.002753 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0d053-7d0f-466b-9e70-492273261b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:48.003013 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0d053-7d0f-466b-9e70-492273261b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:48.003284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0d053-7d0f-466b-9e70-492273261b80 admin admin] PATH_INFO: `/v3/users/84607cc4055a4a61b32ca641c3c46318` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:48.004878 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0d053-7d0f-466b-9e70-492273261b80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84607cc4055a4a61b32ca641c3c46318)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:48.007006 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0d053-7d0f-466b-9e70-492273261b80 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:48.055689 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-61b0d053-7d0f-466b-9e70-492273261b80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84607cc4055a4a61b32ca641c3c46318'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:48.056481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-61b0d053-7d0f-466b-9e70-492273261b80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84607cc4055a4a61b32ca641c3c46318'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:48.065525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-61b0d053-7d0f-466b-9e70-492273261b80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84607cc4055a4a61b32ca641c3c46318'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:48.080208 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-61b0d053-7d0f-466b-9e70-492273261b80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84607cc4055a4a61b32ca641c3c46318'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:48.096138 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-61b0d053-7d0f-466b-9e70-492273261b80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84607cc4055a4a61b32ca641c3c46318'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:48.102372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1335/2643] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:39:47 2022] DELETE /identity/v3/users/84607cc4055a4a61b32ca641c3c46318 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:48.405234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:48.406638 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:48.420174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64729f8-6deb-4524-86ab-29eaf0bc4ce8 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:48.432311 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:48.479246 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb134d5-12d0-4be3-b3d9-1a4b892833ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:48.479898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb134d5-12d0-4be3-b3d9-1a4b892833ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:48.480174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb134d5-12d0-4be3-b3d9-1a4b892833ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:48.480443 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb134d5-12d0-4be3-b3d9-1a4b892833ef admin admin] PATH_INFO: `/v3/projects/9091e79697414317bd2b3b75eecc6159` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:48.482160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb134d5-12d0-4be3-b3d9-1a4b892833ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=9091e79697414317bd2b3b75eecc6159)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:48.484078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb134d5-12d0-4be3-b3d9-1a4b892833ef admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:48.528202 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-bdb134d5-12d0-4be3-b3d9-1a4b892833ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9091e79697414317bd2b3b75eecc6159', 'request_id': 'req-bdb134d5-12d0-4be3-b3d9-1a4b892833ef'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:48.534051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-bdb134d5-12d0-4be3-b3d9-1a4b892833ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9091e79697414317bd2b3b75eecc6159', 'request_id': 'req-bdb134d5-12d0-4be3-b3d9-1a4b892833ef'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:48.542215 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-bdb134d5-12d0-4be3-b3d9-1a4b892833ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9091e79697414317bd2b3b75eecc6159', 'request_id': 'req-bdb134d5-12d0-4be3-b3d9-1a4b892833ef'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:48.553095 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-bdb134d5-12d0-4be3-b3d9-1a4b892833ef admin admin] The token cache is being invalidate because project 9091e79697414317bd2b3b75eecc6159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:39:48.553382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-bdb134d5-12d0-4be3-b3d9-1a4b892833ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:48.554008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-bdb134d5-12d0-4be3-b3d9-1a4b892833ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:39:48.555452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1309/2644] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:39:48 2022] DELETE /identity/v3/projects/9091e79697414317bd2b3b75eecc6159 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:48.560750 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f691ac-d3e1-45f2-976d-fbd60c370163 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:48.561045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f691ac-d3e1-45f2-976d-fbd60c370163 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:48.561356 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f691ac-d3e1-45f2-976d-fbd60c370163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:48.581246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-93f691ac-d3e1-45f2-976d-fbd60c370163 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:48.582454 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-93f691ac-d3e1-45f2-976d-fbd60c370163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:48.618808 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:48.648139 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1336/2645] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:48.656602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b410da8-c69a-4515-acb8-50363dce0961 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:48.656905 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b410da8-c69a-4515-acb8-50363dce0961 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:48.657176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b410da8-c69a-4515-acb8-50363dce0961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:48.676567 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-7b410da8-c69a-4515-acb8-50363dce0961 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:48.677754 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7b410da8-c69a-4515-acb8-50363dce0961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:48.686378 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1310/2646] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:48.690495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f691ac-d3e1-45f2-976d-fbd60c370163 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:48.701495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f691ac-d3e1-45f2-976d-fbd60c370163 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:48.706248 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b492c50-999e-48c2-9a45-9e68db466ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:48.706856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b492c50-999e-48c2-9a45-9e68db466ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:48.707257 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b492c50-999e-48c2-9a45-9e68db466ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:48.707541 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b492c50-999e-48c2-9a45-9e68db466ec6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:48.708222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b492c50-999e-48c2-9a45-9e68db466ec6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:39:48.708529 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b492c50-999e-48c2-9a45-9e68db466ec6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:48.710169 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b492c50-999e-48c2-9a45-9e68db466ec6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:48.723490 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1337/2647] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:39:48.728414 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b410da8-c69a-4515-acb8-50363dce0961 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:48.738253 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b410da8-c69a-4515-acb8-50363dce0961 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:48.745617 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c5e38-1fa4-4bd0-b707-001a07bce11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:48.746239 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c5e38-1fa4-4bd0-b707-001a07bce11b admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:48.746517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c5e38-1fa4-4bd0-b707-001a07bce11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:48.746787 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85c5e38-1fa4-4bd0-b707-001a07bce11b admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:48.749839 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c5e38-1fa4-4bd0-b707-001a07bce11b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:48.752380 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85c5e38-1fa4-4bd0-b707-001a07bce11b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:48.787526 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1311/2648] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:48 2022] POST /identity/v3/projects => generated 399 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:48.791665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b492c50-999e-48c2-9a45-9e68db466ec6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:48.802379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b492c50-999e-48c2-9a45-9e68db466ec6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:48.876538 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0f085-68ca-4dbe-932e-7792815ee563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:48.877164 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0f085-68ca-4dbe-932e-7792815ee563 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:48.877453 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0f085-68ca-4dbe-932e-7792815ee563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:48.877733 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0f085-68ca-4dbe-932e-7792815ee563 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:48.878410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0f085-68ca-4dbe-932e-7792815ee563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:48.880441 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0f085-68ca-4dbe-932e-7792815ee563 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:48.907010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1338/2649] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:48 2022] POST /identity/v3/users => generated 300 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:48.908093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c5e38-1fa4-4bd0-b707-001a07bce11b admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:48.919467 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85c5e38-1fa4-4bd0-b707-001a07bce11b admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:48.924471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da49835-c532-4acf-87bd-731d60b8aaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:48.925092 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da49835-c532-4acf-87bd-731d60b8aaf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:48.925360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da49835-c532-4acf-87bd-731d60b8aaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:48.925625 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da49835-c532-4acf-87bd-731d60b8aaf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:48.926263 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da49835-c532-4acf-87bd-731d60b8aaf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:48.928740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da49835-c532-4acf-87bd-731d60b8aaf0 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:48.943013 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1312/2650] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:48.947285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0f085-68ca-4dbe-932e-7792815ee563 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:48.958768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0f085-68ca-4dbe-932e-7792815ee563 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:48.970535 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce56f17-982b-4b39-b2fb-e388d1b3ae71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:48.971573 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce56f17-982b-4b39-b2fb-e388d1b3ae71 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:48.971730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce56f17-982b-4b39-b2fb-e388d1b3ae71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:48.971995 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce56f17-982b-4b39-b2fb-e388d1b3ae71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:48.972797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce56f17-982b-4b39-b2fb-e388d1b3ae71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:48.974459 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce56f17-982b-4b39-b2fb-e388d1b3ae71 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:48.987792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1339/2651] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:48.991822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da49835-c532-4acf-87bd-731d60b8aaf0 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:49.003834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da49835-c532-4acf-87bd-731d60b8aaf0 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:49.019465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536522d-5c40-40e4-bfb7-98c5b2b1a775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:49.020030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536522d-5c40-40e4-bfb7-98c5b2b1a775 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:49.020314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536522d-5c40-40e4-bfb7-98c5b2b1a775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:49.020585 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536522d-5c40-40e4-bfb7-98c5b2b1a775 admin admin] PATH_INFO: `/v3/projects/f263a05d46a34220ae5e529ac413e4a9/users/9f654defa4374ece9ad631fe635cf5e1/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:49.038433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536522d-5c40-40e4-bfb7-98c5b2b1a775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f263a05d46a34220ae5e529ac413e4a9, user_id=9f654defa4374ece9ad631fe635cf5e1, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:49.041262 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536522d-5c40-40e4-bfb7-98c5b2b1a775 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:49.057697 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1313/2652] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:48 2022] PUT /identity/v3/projects/f263a05d46a34220ae5e529ac413e4a9/users/9f654defa4374ece9ad631fe635cf5e1/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:49.069880 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecf15c-b40c-4ff3-8947-2aeb475b611e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:49.070168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecf15c-b40c-4ff3-8947-2aeb475b611e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:49.070444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ecf15c-b40c-4ff3-8947-2aeb475b611e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:49.098182 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-30ecf15c-b40c-4ff3-8947-2aeb475b611e None None] MFA Rules not processed for user `9f654defa4374ece9ad631fe635cf5e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:49.099395 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-30ecf15c-b40c-4ff3-8947-2aeb475b611e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:49.165083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1340/2653] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:49 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:49.170454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536522d-5c40-40e4-bfb7-98c5b2b1a775 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:49.216669 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:49.236702 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536522d-5c40-40e4-bfb7-98c5b2b1a775 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:49.241418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d4406-fa26-48c0-b5bd-87b18928dd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:49.242024 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d4406-fa26-48c0-b5bd-87b18928dd02 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:49.242309 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d4406-fa26-48c0-b5bd-87b18928dd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:49.242686 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d4406-fa26-48c0-b5bd-87b18928dd02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:49.243210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d4406-fa26-48c0-b5bd-87b18928dd02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:49.245144 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d4406-fa26-48c0-b5bd-87b18928dd02 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:49.276382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1314/2654] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:39:49 2022] POST /identity/v3/projects => generated 399 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:49.277687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecf15c-b40c-4ff3-8947-2aeb475b611e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:49.289988 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ecf15c-b40c-4ff3-8947-2aeb475b611e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:49.328098 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:49.358278 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7dbdf-b55e-4e37-9056-6bb0e33e23d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:49.358942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7dbdf-b55e-4e37-9056-6bb0e33e23d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:49.359374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7dbdf-b55e-4e37-9056-6bb0e33e23d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:49.359687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7dbdf-b55e-4e37-9056-6bb0e33e23d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:49.360338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7dbdf-b55e-4e37-9056-6bb0e33e23d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:49.362373 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7dbdf-b55e-4e37-9056-6bb0e33e23d8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:49.393633 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1341/2655] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:39:49 2022] POST /identity/v3/users => generated 306 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:39:49.397901 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d4406-fa26-48c0-b5bd-87b18928dd02 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:49.417956 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d4406-fa26-48c0-b5bd-87b18928dd02 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:49.423398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d0d4a-14b4-4a1e-bdbd-9a5feb5bdc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:49.424040 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d0d4a-14b4-4a1e-bdbd-9a5feb5bdc45 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:49.424325 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d0d4a-14b4-4a1e-bdbd-9a5feb5bdc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:49.424602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d0d4a-14b4-4a1e-bdbd-9a5feb5bdc45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:49.425254 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d0d4a-14b4-4a1e-bdbd-9a5feb5bdc45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:49.426852 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380d0d4a-14b4-4a1e-bdbd-9a5feb5bdc45 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:49.438174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1315/2656] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:39:49.442553 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7dbdf-b55e-4e37-9056-6bb0e33e23d8 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:49.453821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7dbdf-b55e-4e37-9056-6bb0e33e23d8 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:49.458683 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67053700-8f11-434a-99bd-3b405f77bca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:49.459344 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67053700-8f11-434a-99bd-3b405f77bca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:49.459643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67053700-8f11-434a-99bd-3b405f77bca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:49.459940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67053700-8f11-434a-99bd-3b405f77bca6 admin admin] PATH_INFO: `/v3/projects/965cf87102104082bf1be975d88cb9f3/users/6dea7accf24c46e189aa92280ed77332/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:49.476768 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67053700-8f11-434a-99bd-3b405f77bca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=965cf87102104082bf1be975d88cb9f3, user_id=6dea7accf24c46e189aa92280ed77332, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:49.479446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67053700-8f11-434a-99bd-3b405f77bca6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:49.492307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1342/2657] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:49 2022] PUT /identity/v3/projects/965cf87102104082bf1be975d88cb9f3/users/6dea7accf24c46e189aa92280ed77332/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:49.497554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d0d4a-14b4-4a1e-bdbd-9a5feb5bdc45 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:49.509729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d0d4a-14b4-4a1e-bdbd-9a5feb5bdc45 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:49.567302 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7a8a3-b893-4fe2-98df-15d6d56361e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:49.567932 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7a8a3-b893-4fe2-98df-15d6d56361e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:49.568217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7a8a3-b893-4fe2-98df-15d6d56361e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:49.568494 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7a8a3-b893-4fe2-98df-15d6d56361e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:49.569133 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7a8a3-b893-4fe2-98df-15d6d56361e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:49.570725 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e7a8a3-b893-4fe2-98df-15d6d56361e7 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:49.581256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1316/2658] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:39:49 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:39:49.585495 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67053700-8f11-434a-99bd-3b405f77bca6 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:49.596562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67053700-8f11-434a-99bd-3b405f77bca6 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:49.601149 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70ed81-d105-4c4e-a851-ef10b95d6ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:49.601786 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70ed81-d105-4c4e-a851-ef10b95d6ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:49.602063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70ed81-d105-4c4e-a851-ef10b95d6ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:49.602334 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70ed81-d105-4c4e-a851-ef10b95d6ccb admin admin] PATH_INFO: `/v3/projects/965cf87102104082bf1be975d88cb9f3/users/6dea7accf24c46e189aa92280ed77332/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:49.604244 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70ed81-d105-4c4e-a851-ef10b95d6ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=965cf87102104082bf1be975d88cb9f3, user_id=6dea7accf24c46e189aa92280ed77332, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:49.607067 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70ed81-d105-4c4e-a851-ef10b95d6ccb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:49.617154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1343/2659] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:39:49 2022] PUT /identity/v3/projects/965cf87102104082bf1be975d88cb9f3/users/6dea7accf24c46e189aa92280ed77332/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:39:49.627824 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5295dc-1e0b-4ba8-91aa-ce6f6ee4017c None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:49.628179 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5295dc-1e0b-4ba8-91aa-ce6f6ee4017c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:49.628461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5295dc-1e0b-4ba8-91aa-ce6f6ee4017c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:49.655731 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-dc5295dc-1e0b-4ba8-91aa-ce6f6ee4017c None None] MFA Rules not processed for user `6dea7accf24c46e189aa92280ed77332`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:39:49.656940 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-dc5295dc-1e0b-4ba8-91aa-ce6f6ee4017c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:49.723243 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1317/2660] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:39:49 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:39:49.735070 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70ed81-d105-4c4e-a851-ef10b95d6ccb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:49.736851 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-3c70ed81-d105-4c4e-a851-ef10b95d6ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:39:49.794705 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70ed81-d105-4c4e-a851-ef10b95d6ccb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:49.798356 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e5e73-a8c5-4eb9-aa10-fd1d1b339507 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:49.798951 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e5e73-a8c5-4eb9-aa10-fd1d1b339507 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:49.799271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e5e73-a8c5-4eb9-aa10-fd1d1b339507 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:49.799549 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e5e73-a8c5-4eb9-aa10-fd1d1b339507 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:49.808714 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e5e73-a8c5-4eb9-aa10-fd1d1b339507 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:49.810857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e5e73-a8c5-4eb9-aa10-fd1d1b339507 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:49.820283 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1344/2661] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:49 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:49.898060 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5295dc-1e0b-4ba8-91aa-ce6f6ee4017c None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:39:49.908045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5295dc-1e0b-4ba8-91aa-ce6f6ee4017c None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:39:49.911926 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ca00a-0ace-41e0-8989-dd3d2f881bc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:39:49.912514 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ca00a-0ace-41e0-8989-dd3d2f881bc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:39:49.912791 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ca00a-0ace-41e0-8989-dd3d2f881bc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:39:49.913059 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ca00a-0ace-41e0-8989-dd3d2f881bc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:39:49.946053 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:49.970858 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ca00a-0ace-41e0-8989-dd3d2f881bc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:39:49.972331 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ca00a-0ace-41e0-8989-dd3d2f881bc4 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:39:49.981025 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1318/2662] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:39:49 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:39:59.816063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=55,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:39:59.977139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=11 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:01.812478 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188eb78a-6f58-410f-9f9e-f7533dfe5435 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:01.813073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188eb78a-6f58-410f-9f9e-f7533dfe5435 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:01.813073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188eb78a-6f58-410f-9f9e-f7533dfe5435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:01.833035 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-188eb78a-6f58-410f-9f9e-f7533dfe5435 None None] MFA Rules not processed for user `9c40bc0027e94421a1da7b8016307b67`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:01.834174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-188eb78a-6f58-410f-9f9e-f7533dfe5435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:01.898099 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1345/2663] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:02.177079 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ca00a-0ace-41e0-8989-dd3d2f881bc4 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:02.181741 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-3f5ca00a-0ace-41e0-8989-dd3d2f881bc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:02.194982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ca00a-0ace-41e0-8989-dd3d2f881bc4 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:02.198679 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba6f78-ccd3-4c7e-a5da-fe1fbcd073d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:02.199223 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba6f78-ccd3-4c7e-a5da-fe1fbcd073d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:02.199801 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba6f78-ccd3-4c7e-a5da-fe1fbcd073d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:02.200509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba6f78-ccd3-4c7e-a5da-fe1fbcd073d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:02.201984 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-0dba6f78-ccd3-4c7e-a5da-fe1fbcd073d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:02.296105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba6f78-ccd3-4c7e-a5da-fe1fbcd073d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:02.297760 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dba6f78-ccd3-4c7e-a5da-fe1fbcd073d5 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:02.313400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1319/2664] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:03.375170 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188eb78a-6f58-410f-9f9e-f7533dfe5435 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:03.376624 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-188eb78a-6f58-410f-9f9e-f7533dfe5435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:03.441343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188eb78a-6f58-410f-9f9e-f7533dfe5435 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:03.445868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46321617-f270-4380-895d-f854a1cb0651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:03.446490 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46321617-f270-4380-895d-f854a1cb0651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:03.446771 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46321617-f270-4380-895d-f854a1cb0651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:03.447043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46321617-f270-4380-895d-f854a1cb0651 admin admin] PATH_INFO: `/v3/users/5d82dc4020f84eb6bb0ae5ccbafae68a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:03.448169 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46321617-f270-4380-895d-f854a1cb0651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d82dc4020f84eb6bb0ae5ccbafae68a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:03.450349 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46321617-f270-4380-895d-f854a1cb0651 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:03.495996 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-46321617-f270-4380-895d-f854a1cb0651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d82dc4020f84eb6bb0ae5ccbafae68a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:03.496649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-46321617-f270-4380-895d-f854a1cb0651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d82dc4020f84eb6bb0ae5ccbafae68a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:03.503779 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-46321617-f270-4380-895d-f854a1cb0651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d82dc4020f84eb6bb0ae5ccbafae68a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:03.511435 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-46321617-f270-4380-895d-f854a1cb0651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d82dc4020f84eb6bb0ae5ccbafae68a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:03.524712 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-46321617-f270-4380-895d-f854a1cb0651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d82dc4020f84eb6bb0ae5ccbafae68a'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:03.531320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1346/2665] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:03 2022] DELETE /identity/v3/users/5d82dc4020f84eb6bb0ae5ccbafae68a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:03.536010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734577a-d7de-4cbc-a885-fee13badc6ec None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:03.536475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734577a-d7de-4cbc-a885-fee13badc6ec None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:03.536606 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734577a-d7de-4cbc-a885-fee13badc6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:03.555369 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-3734577a-d7de-4cbc-a885-fee13badc6ec None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:03.556611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-3734577a-d7de-4cbc-a885-fee13badc6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:03.624995 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1320/2666] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:03.632921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46321617-f270-4380-895d-f854a1cb0651 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:03.633704 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-46321617-f270-4380-895d-f854a1cb0651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:03.664558 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:03.696709 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46321617-f270-4380-895d-f854a1cb0651 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:03.701879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5748e-0314-41bb-80c1-2ac6139b3db1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:03.702489 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5748e-0314-41bb-80c1-2ac6139b3db1 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:03.702775 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5748e-0314-41bb-80c1-2ac6139b3db1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:03.703053 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5748e-0314-41bb-80c1-2ac6139b3db1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:03.714182 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5748e-0314-41bb-80c1-2ac6139b3db1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:03.715643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5748e-0314-41bb-80c1-2ac6139b3db1 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:03.727352 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1347/2667] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:03.774302 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734577a-d7de-4cbc-a885-fee13badc6ec None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:03.775796 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-3734577a-d7de-4cbc-a885-fee13badc6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:03.788677 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734577a-d7de-4cbc-a885-fee13badc6ec None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:03.794609 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d21a71-993a-47b9-87f0-c62fdf35725c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:03.796035 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d21a71-993a-47b9-87f0-c62fdf35725c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:03.796380 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d21a71-993a-47b9-87f0-c62fdf35725c admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:03.796693 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d21a71-993a-47b9-87f0-c62fdf35725c admin admin] PATH_INFO: `/v3/users/9a2d5eb11fd541ddb0b2817312c3acaf` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:03.797952 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d21a71-993a-47b9-87f0-c62fdf35725c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a2d5eb11fd541ddb0b2817312c3acaf)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:03.800126 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d21a71-993a-47b9-87f0-c62fdf35725c admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:03.853921 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-07d21a71-993a-47b9-87f0-c62fdf35725c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a2d5eb11fd541ddb0b2817312c3acaf'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:03.854774 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-07d21a71-993a-47b9-87f0-c62fdf35725c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a2d5eb11fd541ddb0b2817312c3acaf'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:03.855481 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-07d21a71-993a-47b9-87f0-c62fdf35725c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a2d5eb11fd541ddb0b2817312c3acaf'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:03.866220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-07d21a71-993a-47b9-87f0-c62fdf35725c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a2d5eb11fd541ddb0b2817312c3acaf'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:03.894671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-07d21a71-993a-47b9-87f0-c62fdf35725c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a2d5eb11fd541ddb0b2817312c3acaf'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:03.927969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1321/2668] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:03 2022] DELETE /identity/v3/users/9a2d5eb11fd541ddb0b2817312c3acaf => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:03.932288 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc012e-7830-457d-9e3d-f1f685aa39f7 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:03.932578 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc012e-7830-457d-9e3d-f1f685aa39f7 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:03.932851 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc012e-7830-457d-9e3d-f1f685aa39f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:03.955868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-bcdc012e-7830-457d-9e3d-f1f685aa39f7 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:03.957744 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bcdc012e-7830-457d-9e3d-f1f685aa39f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:04.017696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1348/2669] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:04.026135 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d21a71-993a-47b9-87f0-c62fdf35725c admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:04.027556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-07d21a71-993a-47b9-87f0-c62fdf35725c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:04.029976 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc012e-7830-457d-9e3d-f1f685aa39f7 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:04.031313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-bcdc012e-7830-457d-9e3d-f1f685aa39f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:04.050607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d21a71-993a-47b9-87f0-c62fdf35725c admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:04.055603 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4054117-e856-42d3-aaa9-a0d7e6b47435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:04.056279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4054117-e856-42d3-aaa9-a0d7e6b47435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:04.056565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4054117-e856-42d3-aaa9-a0d7e6b47435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:04.056951 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4054117-e856-42d3-aaa9-a0d7e6b47435 admin admin] PATH_INFO: `/v3/projects/9ff095a5b52f4f7088f0331072c0d2f6` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:04.057971 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4054117-e856-42d3-aaa9-a0d7e6b47435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ff095a5b52f4f7088f0331072c0d2f6)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:04.060454 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4054117-e856-42d3-aaa9-a0d7e6b47435 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:04.067003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:04.094913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc012e-7830-457d-9e3d-f1f685aa39f7 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:04.099173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d899e-139d-43aa-b6c2-fad131461e52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:04.099860 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d899e-139d-43aa-b6c2-fad131461e52 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:04.100139 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d899e-139d-43aa-b6c2-fad131461e52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:04.100465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d899e-139d-43aa-b6c2-fad131461e52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:04.109706 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d899e-139d-43aa-b6c2-fad131461e52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:04.111174 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d899e-139d-43aa-b6c2-fad131461e52 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:04.111989 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d4054117-e856-42d3-aaa9-a0d7e6b47435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ff095a5b52f4f7088f0331072c0d2f6', 'request_id': 'req-d4054117-e856-42d3-aaa9-a0d7e6b47435'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.118809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d4054117-e856-42d3-aaa9-a0d7e6b47435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ff095a5b52f4f7088f0331072c0d2f6', 'request_id': 'req-d4054117-e856-42d3-aaa9-a0d7e6b47435'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.120717 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1349/2670] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:04.127636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d4054117-e856-42d3-aaa9-a0d7e6b47435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ff095a5b52f4f7088f0331072c0d2f6', 'request_id': 'req-d4054117-e856-42d3-aaa9-a0d7e6b47435'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.140046 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d4054117-e856-42d3-aaa9-a0d7e6b47435 admin admin] The token cache is being invalidate because project 9ff095a5b52f4f7088f0331072c0d2f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:04.140338 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d4054117-e856-42d3-aaa9-a0d7e6b47435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.141079 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d4054117-e856-42d3-aaa9-a0d7e6b47435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.142613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1322/2671] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:04 2022] DELETE /identity/v3/projects/9ff095a5b52f4f7088f0331072c0d2f6 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:04.146579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d899e-139d-43aa-b6c2-fad131461e52 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:04.156851 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d899e-139d-43aa-b6c2-fad131461e52 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:04.219245 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a4243-b11b-40a7-be9f-a681ace1b91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:04.220073 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a4243-b11b-40a7-be9f-a681ace1b91d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:04.220384 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a4243-b11b-40a7-be9f-a681ace1b91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:04.220648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a4243-b11b-40a7-be9f-a681ace1b91d admin admin] PATH_INFO: `/v3/users/4fc5431754cb476bade9d7a1dfbabe9e` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:04.221936 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a4243-b11b-40a7-be9f-a681ace1b91d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fc5431754cb476bade9d7a1dfbabe9e)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:04.223942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a4243-b11b-40a7-be9f-a681ace1b91d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:04.241678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:04.280891 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-881a4243-b11b-40a7-be9f-a681ace1b91d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fc5431754cb476bade9d7a1dfbabe9e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.281576 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-881a4243-b11b-40a7-be9f-a681ace1b91d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fc5431754cb476bade9d7a1dfbabe9e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.294653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-881a4243-b11b-40a7-be9f-a681ace1b91d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fc5431754cb476bade9d7a1dfbabe9e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.309605 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-881a4243-b11b-40a7-be9f-a681ace1b91d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fc5431754cb476bade9d7a1dfbabe9e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.323045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-881a4243-b11b-40a7-be9f-a681ace1b91d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fc5431754cb476bade9d7a1dfbabe9e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.330566 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1350/2672] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:04 2022] DELETE /identity/v3/users/4fc5431754cb476bade9d7a1dfbabe9e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:04.421298 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4054117-e856-42d3-aaa9-a0d7e6b47435 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:04.422981 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d4054117-e856-42d3-aaa9-a0d7e6b47435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:04.438070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4054117-e856-42d3-aaa9-a0d7e6b47435 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:04.503335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196e66b-7a95-4830-9f98-e4c1e3bdf448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:04.504111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196e66b-7a95-4830-9f98-e4c1e3bdf448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:04.504396 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196e66b-7a95-4830-9f98-e4c1e3bdf448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:04.504670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196e66b-7a95-4830-9f98-e4c1e3bdf448 admin admin] PATH_INFO: `/v3/projects/8b15f2e9825749f1af329d4341894f93` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:04.505821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196e66b-7a95-4830-9f98-e4c1e3bdf448 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b15f2e9825749f1af329d4341894f93)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:04.507810 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196e66b-7a95-4830-9f98-e4c1e3bdf448 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:04.556512 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3196e66b-7a95-4830-9f98-e4c1e3bdf448 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b15f2e9825749f1af329d4341894f93', 'request_id': 'req-3196e66b-7a95-4830-9f98-e4c1e3bdf448'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.563113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3196e66b-7a95-4830-9f98-e4c1e3bdf448 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b15f2e9825749f1af329d4341894f93', 'request_id': 'req-3196e66b-7a95-4830-9f98-e4c1e3bdf448'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.571608 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3196e66b-7a95-4830-9f98-e4c1e3bdf448 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b15f2e9825749f1af329d4341894f93', 'request_id': 'req-3196e66b-7a95-4830-9f98-e4c1e3bdf448'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.584527 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3196e66b-7a95-4830-9f98-e4c1e3bdf448 admin admin] The token cache is being invalidate because project 8b15f2e9825749f1af329d4341894f93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:04.584821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3196e66b-7a95-4830-9f98-e4c1e3bdf448 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.585475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-3196e66b-7a95-4830-9f98-e4c1e3bdf448 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.587363 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1323/2673] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:04 2022] DELETE /identity/v3/projects/8b15f2e9825749f1af329d4341894f93 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:04.591462 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a4243-b11b-40a7-be9f-a681ace1b91d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:04.592940 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-881a4243-b11b-40a7-be9f-a681ace1b91d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:04.605723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a4243-b11b-40a7-be9f-a681ace1b91d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:04.624727 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196e66b-7a95-4830-9f98-e4c1e3bdf448 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:04.627019 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-3196e66b-7a95-4830-9f98-e4c1e3bdf448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:04.661463 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:04.678879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a49bdc-af06-475b-85c4-9f425d11bce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:04.679752 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a49bdc-af06-475b-85c4-9f425d11bce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:04.680129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a49bdc-af06-475b-85c4-9f425d11bce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:04.680477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a49bdc-af06-475b-85c4-9f425d11bce5 admin admin] PATH_INFO: `/v3/users/3445d981e433449b8ca8988ac7e7ceba` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:04.681727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a49bdc-af06-475b-85c4-9f425d11bce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3445d981e433449b8ca8988ac7e7ceba)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:04.684092 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a49bdc-af06-475b-85c4-9f425d11bce5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:04.705359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196e66b-7a95-4830-9f98-e4c1e3bdf448 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:04.711212 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d3f08b-564b-4533-9853-522d09f03e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:04.711878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3f08b-564b-4533-9853-522d09f03e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:04.712169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3f08b-564b-4533-9853-522d09f03e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:04.712428 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3f08b-564b-4533-9853-522d09f03e13 admin admin] PATH_INFO: `/v3/projects/e63b32aebb7046d19116727d076c7904` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:04.713657 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d3f08b-564b-4533-9853-522d09f03e13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e63b32aebb7046d19116727d076c7904)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:04.715636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d3f08b-564b-4533-9853-522d09f03e13 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:04.736811 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c6a49bdc-af06-475b-85c4-9f425d11bce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3445d981e433449b8ca8988ac7e7ceba'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.737482 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c6a49bdc-af06-475b-85c4-9f425d11bce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3445d981e433449b8ca8988ac7e7ceba'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.745989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c6a49bdc-af06-475b-85c4-9f425d11bce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3445d981e433449b8ca8988ac7e7ceba'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.754573 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c6a49bdc-af06-475b-85c4-9f425d11bce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3445d981e433449b8ca8988ac7e7ceba'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.769664 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-76d3f08b-564b-4533-9853-522d09f03e13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e63b32aebb7046d19116727d076c7904', 'request_id': 'req-76d3f08b-564b-4533-9853-522d09f03e13'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.773246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c6a49bdc-af06-475b-85c4-9f425d11bce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3445d981e433449b8ca8988ac7e7ceba'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.776606 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-76d3f08b-564b-4533-9853-522d09f03e13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e63b32aebb7046d19116727d076c7904', 'request_id': 'req-76d3f08b-564b-4533-9853-522d09f03e13'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.782117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1351/2674] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:04 2022] DELETE /identity/v3/users/3445d981e433449b8ca8988ac7e7ceba => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:04.786086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-76d3f08b-564b-4533-9853-522d09f03e13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e63b32aebb7046d19116727d076c7904', 'request_id': 'req-76d3f08b-564b-4533-9853-522d09f03e13'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.799018 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-76d3f08b-564b-4533-9853-522d09f03e13 admin admin] The token cache is being invalidate because project e63b32aebb7046d19116727d076c7904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:04.799380 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-76d3f08b-564b-4533-9853-522d09f03e13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.800144 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-76d3f08b-564b-4533-9853-522d09f03e13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:04.801990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1324/2675] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:04 2022] DELETE /identity/v3/projects/e63b32aebb7046d19116727d076c7904 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:04.806757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ac174-5f75-46d2-8ad7-3891aa691169 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:04.807132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ac174-5f75-46d2-8ad7-3891aa691169 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:04.807533 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ac174-5f75-46d2-8ad7-3891aa691169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:04.833272 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6f8ac174-5f75-46d2-8ad7-3891aa691169 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:04.834564 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6f8ac174-5f75-46d2-8ad7-3891aa691169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:04.839519 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:04.906467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1352/2676] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:04.914066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311550f-d8ea-40fd-9182-92c1908f0eee None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:04.914515 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311550f-d8ea-40fd-9182-92c1908f0eee None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:04.914751 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311550f-d8ea-40fd-9182-92c1908f0eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:04.953582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-b311550f-d8ea-40fd-9182-92c1908f0eee None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:04.954815 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-b311550f-d8ea-40fd-9182-92c1908f0eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:04.977491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1325/2677] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:04.978643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ac174-5f75-46d2-8ad7-3891aa691169 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:04.989965 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ac174-5f75-46d2-8ad7-3891aa691169 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:04.996031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ab353-ab03-49fa-b4f1-1c0f7ee08b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:04.996755 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ab353-ab03-49fa-b4f1-1c0f7ee08b36 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:04.997219 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ab353-ab03-49fa-b4f1-1c0f7ee08b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:04.997864 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ab353-ab03-49fa-b4f1-1c0f7ee08b36 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:04.998656 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ab353-ab03-49fa-b4f1-1c0f7ee08b36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:04.999085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ab353-ab03-49fa-b4f1-1c0f7ee08b36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.001625 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ab353-ab03-49fa-b4f1-1c0f7ee08b36 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.015229 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1353/2678] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:40:05.019611 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311550f-d8ea-40fd-9182-92c1908f0eee None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.029333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311550f-d8ea-40fd-9182-92c1908f0eee None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.034002 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b4407-d6e3-4106-884f-35432badf89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.034602 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b4407-d6e3-4106-884f-35432badf89e admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.034878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b4407-d6e3-4106-884f-35432badf89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.035150 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b4407-d6e3-4106-884f-35432badf89e admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.035826 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b4407-d6e3-4106-884f-35432badf89e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.037703 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4b4407-d6e3-4106-884f-35432badf89e admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.071928 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1326/2679] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:05 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:05.079649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ab353-ab03-49fa-b4f1-1c0f7ee08b36 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.092460 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ab353-ab03-49fa-b4f1-1c0f7ee08b36 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.105721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b4407-d6e3-4106-884f-35432badf89e admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.108195 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-9e4b4407-d6e3-4106-884f-35432badf89e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:05.157121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd7a66-e1ac-48e4-a42e-c2379f6b315b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.157740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd7a66-e1ac-48e4-a42e-c2379f6b315b admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.158128 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd7a66-e1ac-48e4-a42e-c2379f6b315b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.158713 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd7a66-e1ac-48e4-a42e-c2379f6b315b admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.159173 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd7a66-e1ac-48e4-a42e-c2379f6b315b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.161236 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd7a66-e1ac-48e4-a42e-c2379f6b315b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.183583 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b4407-d6e3-4106-884f-35432badf89e admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.189279 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1354/2680] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:05 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:05.189444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12181c84-d8e6-419e-8e67-e0c8441ae56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.189623 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12181c84-d8e6-419e-8e67-e0c8441ae56f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.189909 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12181c84-d8e6-419e-8e67-e0c8441ae56f admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.190330 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12181c84-d8e6-419e-8e67-e0c8441ae56f admin admin] PATH_INFO: `/v3/projects/3095403f34394e3e8623d9055f54841e` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.190448 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd7a66-e1ac-48e4-a42e-c2379f6b315b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.191475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12181c84-d8e6-419e-8e67-e0c8441ae56f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3095403f34394e3e8623d9055f54841e)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.193411 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12181c84-d8e6-419e-8e67-e0c8441ae56f admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.201500 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd7a66-e1ac-48e4-a42e-c2379f6b315b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.206342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd0f03-ea8c-403c-a115-b1a43ebc0fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.206993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd0f03-ea8c-403c-a115-b1a43ebc0fef admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.207239 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd0f03-ea8c-403c-a115-b1a43ebc0fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.207522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd0f03-ea8c-403c-a115-b1a43ebc0fef admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.208206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd0f03-ea8c-403c-a115-b1a43ebc0fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.209795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd0f03-ea8c-403c-a115-b1a43ebc0fef admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.220586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1355/2681] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:05.225618 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd0f03-ea8c-403c-a115-b1a43ebc0fef admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.227283 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:05.236219 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd0f03-ea8c-403c-a115-b1a43ebc0fef admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.246354 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-12181c84-d8e6-419e-8e67-e0c8441ae56f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3095403f34394e3e8623d9055f54841e', 'request_id': 'req-12181c84-d8e6-419e-8e67-e0c8441ae56f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:05.252534 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-12181c84-d8e6-419e-8e67-e0c8441ae56f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3095403f34394e3e8623d9055f54841e', 'request_id': 'req-12181c84-d8e6-419e-8e67-e0c8441ae56f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:05.261464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-12181c84-d8e6-419e-8e67-e0c8441ae56f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3095403f34394e3e8623d9055f54841e', 'request_id': 'req-12181c84-d8e6-419e-8e67-e0c8441ae56f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:05.272610 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-12181c84-d8e6-419e-8e67-e0c8441ae56f admin admin] The token cache is being invalidate because project 3095403f34394e3e8623d9055f54841e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:05.272901 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-12181c84-d8e6-419e-8e67-e0c8441ae56f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:05.273563 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-12181c84-d8e6-419e-8e67-e0c8441ae56f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:05.275064 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1327/2682] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:05 2022] DELETE /identity/v3/projects/3095403f34394e3e8623d9055f54841e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:05.279911 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9fc20-f8d8-4adf-99c6-ca8c5039779c None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.280198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9fc20-f8d8-4adf-99c6-ca8c5039779c None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.280498 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b9fc20-f8d8-4adf-99c6-ca8c5039779c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.297253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fac24-ef82-4117-974e-b105c6c95e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.298023 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fac24-ef82-4117-974e-b105c6c95e3b admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.298375 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fac24-ef82-4117-974e-b105c6c95e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.298687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fac24-ef82-4117-974e-b105c6c95e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.299589 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fac24-ef82-4117-974e-b105c6c95e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.301428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fac24-ef82-4117-974e-b105c6c95e3b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.301614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-31b9fc20-f8d8-4adf-99c6-ca8c5039779c None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:05.302845 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-31b9fc20-f8d8-4adf-99c6-ca8c5039779c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:05.311547 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1328/2683] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:05.315016 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1356/2684] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:05.316527 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9fc20-f8d8-4adf-99c6-ca8c5039779c None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.317947 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-31b9fc20-f8d8-4adf-99c6-ca8c5039779c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:05.319545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b126d-982b-4a4e-92d3-cca10b84fbf4 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.319861 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b126d-982b-4a4e-92d3-cca10b84fbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.320143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b126d-982b-4a4e-92d3-cca10b84fbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.330383 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b9fc20-f8d8-4adf-99c6-ca8c5039779c None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.335011 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87d6b6-e8bf-44c6-8dce-36166f2443a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.335686 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87d6b6-e8bf-44c6-8dce-36166f2443a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.335972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87d6b6-e8bf-44c6-8dce-36166f2443a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.336271 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87d6b6-e8bf-44c6-8dce-36166f2443a6 admin admin] PATH_INFO: `/v3/projects/6a8bc5bd33e0432491df4938d4193db6/users/2938858bcc7d445096cf884d107fdb9f/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.340722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-153b126d-982b-4a4e-92d3-cca10b84fbf4 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:05.342474 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-153b126d-982b-4a4e-92d3-cca10b84fbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:05.350469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1357/2685] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:05.351291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87d6b6-e8bf-44c6-8dce-36166f2443a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8bc5bd33e0432491df4938d4193db6, user_id=2938858bcc7d445096cf884d107fdb9f, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.354063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87d6b6-e8bf-44c6-8dce-36166f2443a6 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.354613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b126d-982b-4a4e-92d3-cca10b84fbf4 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.360937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:05.365319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b126d-982b-4a4e-92d3-cca10b84fbf4 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.369193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1329/2686] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:05 2022] PUT /identity/v3/projects/6a8bc5bd33e0432491df4938d4193db6/users/2938858bcc7d445096cf884d107fdb9f/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:05.379340 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcc4bf-028a-4df1-8220-d1c11ecf6cdc None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.379657 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcc4bf-028a-4df1-8220-d1c11ecf6cdc None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.379937 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcc4bf-028a-4df1-8220-d1c11ecf6cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.408781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-46bcc4bf-028a-4df1-8220-d1c11ecf6cdc None None] MFA Rules not processed for user `2938858bcc7d445096cf884d107fdb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:05.410092 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-46bcc4bf-028a-4df1-8220-d1c11ecf6cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:05.436536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c82c6-1723-4193-9bab-b0acd6510325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.437196 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c82c6-1723-4193-9bab-b0acd6510325 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.437481 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c82c6-1723-4193-9bab-b0acd6510325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.437760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1c82c6-1723-4193-9bab-b0acd6510325 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.438413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c82c6-1723-4193-9bab-b0acd6510325 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:05.438722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c82c6-1723-4193-9bab-b0acd6510325 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.440412 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1c82c6-1723-4193-9bab-b0acd6510325 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.456372 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1358/2687] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:40:05.458083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c82c6-1723-4193-9bab-b0acd6510325 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.470585 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1c82c6-1723-4193-9bab-b0acd6510325 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.476598 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932fce3-ee1a-47f1-8780-34fc924b5029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.477239 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932fce3-ee1a-47f1-8780-34fc924b5029 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.477524 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932fce3-ee1a-47f1-8780-34fc924b5029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.477798 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932fce3-ee1a-47f1-8780-34fc924b5029 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.478428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932fce3-ee1a-47f1-8780-34fc924b5029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.480398 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932fce3-ee1a-47f1-8780-34fc924b5029 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.481434 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1330/2688] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:05.486716 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bcc4bf-028a-4df1-8220-d1c11ecf6cdc None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.497266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bcc4bf-028a-4df1-8220-d1c11ecf6cdc None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.502786 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8bf30-62e2-46f6-b91a-3fa923c00ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.503382 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8bf30-62e2-46f6-b91a-3fa923c00ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.503671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8bf30-62e2-46f6-b91a-3fa923c00ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.503992 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8bf30-62e2-46f6-b91a-3fa923c00ae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.504773 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8bf30-62e2-46f6-b91a-3fa923c00ae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.506688 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8bf30-62e2-46f6-b91a-3fa923c00ae4 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.511709 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1359/2689] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:05 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:05.515917 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932fce3-ee1a-47f1-8780-34fc924b5029 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.541060 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1331/2690] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:05 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:05.548366 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8bf30-62e2-46f6-b91a-3fa923c00ae4 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.588251 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932fce3-ee1a-47f1-8780-34fc924b5029 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.612058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:05.625874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8bf30-62e2-46f6-b91a-3fa923c00ae4 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.633334 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15b557-a7a1-4c7e-ba00-aed0e6e73fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.633999 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15b557-a7a1-4c7e-ba00-aed0e6e73fb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.634289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15b557-a7a1-4c7e-ba00-aed0e6e73fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.634561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15b557-a7a1-4c7e-ba00-aed0e6e73fb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.635215 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15b557-a7a1-4c7e-ba00-aed0e6e73fb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.637353 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15b557-a7a1-4c7e-ba00-aed0e6e73fb1 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.662242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1332/2691] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:05 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:05.662469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760f9df-dff0-48fa-a306-94691c24cbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.662469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760f9df-dff0-48fa-a306-94691c24cbfb admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.662469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760f9df-dff0-48fa-a306-94691c24cbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.662469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760f9df-dff0-48fa-a306-94691c24cbfb admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.663296 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760f9df-dff0-48fa-a306-94691c24cbfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.663530 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15b557-a7a1-4c7e-ba00-aed0e6e73fb1 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.663720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760f9df-dff0-48fa-a306-94691c24cbfb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.676031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15b557-a7a1-4c7e-ba00-aed0e6e73fb1 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.681497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8c5fb-3bb6-4873-a770-7dad26abbae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.682239 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8c5fb-3bb6-4873-a770-7dad26abbae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.682527 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8c5fb-3bb6-4873-a770-7dad26abbae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.683418 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8c5fb-3bb6-4873-a770-7dad26abbae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.683575 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8c5fb-3bb6-4873-a770-7dad26abbae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.685874 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8c5fb-3bb6-4873-a770-7dad26abbae5 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.690568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1360/2692] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:05 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:05.695091 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760f9df-dff0-48fa-a306-94691c24cbfb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.705860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1333/2693] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:05.705860 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8c5fb-3bb6-4873-a770-7dad26abbae5 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.710279 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760f9df-dff0-48fa-a306-94691c24cbfb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.715799 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb7d52-6812-46ea-bb3c-da085cdfcfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.716435 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8c5fb-3bb6-4873-a770-7dad26abbae5 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.716977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb7d52-6812-46ea-bb3c-da085cdfcfa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.716977 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb7d52-6812-46ea-bb3c-da085cdfcfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.717274 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb7d52-6812-46ea-bb3c-da085cdfcfa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.718014 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb7d52-6812-46ea-bb3c-da085cdfcfa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.719767 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb7d52-6812-46ea-bb3c-da085cdfcfa1 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.722919 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ad56e-2434-4f8c-8f7d-23a7a913f53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.723945 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ad56e-2434-4f8c-8f7d-23a7a913f53f admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.724481 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ad56e-2434-4f8c-8f7d-23a7a913f53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.724794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ad56e-2434-4f8c-8f7d-23a7a913f53f admin admin] PATH_INFO: `/v3/projects/9444d921e63940a2aac749d526f61bd4/users/37836f5859134685b1cab124c91f8a96/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.736937 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1361/2694] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:05.738121 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb7d52-6812-46ea-bb3c-da085cdfcfa1 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.742683 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ad56e-2434-4f8c-8f7d-23a7a913f53f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9444d921e63940a2aac749d526f61bd4, user_id=37836f5859134685b1cab124c91f8a96, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.745125 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:05.746078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ad56e-2434-4f8c-8f7d-23a7a913f53f admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.753754 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb7d52-6812-46ea-bb3c-da085cdfcfa1 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.763725 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1334/2695] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:05 2022] PUT /identity/v3/projects/9444d921e63940a2aac749d526f61bd4/users/37836f5859134685b1cab124c91f8a96/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:05.764910 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17de5b3-f51f-4bdf-aff7-9d4047da31e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.765702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17de5b3-f51f-4bdf-aff7-9d4047da31e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.766058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17de5b3-f51f-4bdf-aff7-9d4047da31e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.766425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17de5b3-f51f-4bdf-aff7-9d4047da31e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.767207 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17de5b3-f51f-4bdf-aff7-9d4047da31e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.767595 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ad56e-2434-4f8c-8f7d-23a7a913f53f admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.769300 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17de5b3-f51f-4bdf-aff7-9d4047da31e5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.783883 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1362/2696] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:05.787295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17de5b3-f51f-4bdf-aff7-9d4047da31e5 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.798437 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17de5b3-f51f-4bdf-aff7-9d4047da31e5 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.846805 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ad56e-2434-4f8c-8f7d-23a7a913f53f admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.858517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f019f2-dbf8-4055-80bf-0daad2b31b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.859470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f019f2-dbf8-4055-80bf-0daad2b31b49 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.859470 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f019f2-dbf8-4055-80bf-0daad2b31b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.859732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f019f2-dbf8-4055-80bf-0daad2b31b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.860360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f019f2-dbf8-4055-80bf-0daad2b31b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.860499 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb1e95-7ea5-483b-85a8-94f9d20a4e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.861274 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb1e95-7ea5-483b-85a8-94f9d20a4e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.861625 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb1e95-7ea5-483b-85a8-94f9d20a4e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.861970 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb1e95-7ea5-483b-85a8-94f9d20a4e0c admin admin] PATH_INFO: `/v3/projects/a6ce654b8dc4435981cb395c4b887ae7/users/25fad82cbf944d7db13d6bf33d3297a1/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.862105 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f019f2-dbf8-4055-80bf-0daad2b31b49 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.875111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1335/2697] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:05 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:05.883745 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f019f2-dbf8-4055-80bf-0daad2b31b49 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:05.893014 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb1e95-7ea5-483b-85a8-94f9d20a4e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ce654b8dc4435981cb395c4b887ae7, user_id=25fad82cbf944d7db13d6bf33d3297a1, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.895252 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f019f2-dbf8-4055-80bf-0daad2b31b49 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:05.895918 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb1e95-7ea5-483b-85a8-94f9d20a4e0c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.901042 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38c1a2-fde4-47a4-adb4-9586a4fe1533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:05.902695 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38c1a2-fde4-47a4-adb4-9586a4fe1533 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.903116 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38c1a2-fde4-47a4-adb4-9586a4fe1533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.903762 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38c1a2-fde4-47a4-adb4-9586a4fe1533 admin admin] PATH_INFO: `/v3/projects/9444d921e63940a2aac749d526f61bd4/users/37836f5859134685b1cab124c91f8a96/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.906924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38c1a2-fde4-47a4-adb4-9586a4fe1533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9444d921e63940a2aac749d526f61bd4, user_id=37836f5859134685b1cab124c91f8a96, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:05.911001 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee38c1a2-fde4-47a4-adb4-9586a4fe1533 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:05.921790 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1363/2698] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:05 2022] PUT /identity/v3/projects/a6ce654b8dc4435981cb395c4b887ae7/users/25fad82cbf944d7db13d6bf33d3297a1/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:05.926788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1336/2699] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:05 2022] PUT /identity/v3/projects/9444d921e63940a2aac749d526f61bd4/users/37836f5859134685b1cab124c91f8a96/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:05.929513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964c8b4-de57-429b-b361-a090a6600d23 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.929513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964c8b4-de57-429b-b361-a090a6600d23 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.929513 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964c8b4-de57-429b-b361-a090a6600d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.940868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f7e68-c5b9-4d44-a18f-db698465c071 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:05.942092 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f7e68-c5b9-4d44-a18f-db698465c071 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:05.942279 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f7e68-c5b9-4d44-a18f-db698465c071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:05.958582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a964c8b4-de57-429b-b361-a090a6600d23 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:05.959991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a964c8b4-de57-429b-b361-a090a6600d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:05.978412 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-468f7e68-c5b9-4d44-a18f-db698465c071 None None] MFA Rules not processed for user `37836f5859134685b1cab124c91f8a96`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:05.979929 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-468f7e68-c5b9-4d44-a18f-db698465c071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:06.072465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1364/2700] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:06.076903 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964c8b4-de57-429b-b361-a090a6600d23 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:06.078528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a964c8b4-de57-429b-b361-a090a6600d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:06.099816 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1337/2701] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:05 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:06.103036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f7e68-c5b9-4d44-a18f-db698465c071 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:06.109580 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-468f7e68-c5b9-4d44-a18f-db698465c071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:06.119094 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964c8b4-de57-429b-b361-a090a6600d23 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:06.124160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125d110-1a32-4a9d-9e78-072e931803b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:06.124809 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125d110-1a32-4a9d-9e78-072e931803b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:06.125095 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125d110-1a32-4a9d-9e78-072e931803b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:06.125373 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125d110-1a32-4a9d-9e78-072e931803b6 admin admin] PATH_INFO: `/v3/users/9f654defa4374ece9ad631fe635cf5e1` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:06.126546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b125d110-1a32-4a9d-9e78-072e931803b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f654defa4374ece9ad631fe635cf5e1)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:06.128521 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b125d110-1a32-4a9d-9e78-072e931803b6 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:06.134497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:06.205817 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b125d110-1a32-4a9d-9e78-072e931803b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f654defa4374ece9ad631fe635cf5e1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:06.206812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b125d110-1a32-4a9d-9e78-072e931803b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f654defa4374ece9ad631fe635cf5e1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:06.213436 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f7e68-c5b9-4d44-a18f-db698465c071 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:06.219420 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a0071-666e-4ca2-9914-e026ed50b6ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:06.220154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a0071-666e-4ca2-9914-e026ed50b6ef service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:06.220491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a0071-666e-4ca2-9914-e026ed50b6ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:06.220849 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a0071-666e-4ca2-9914-e026ed50b6ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:06.228319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b125d110-1a32-4a9d-9e78-072e931803b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f654defa4374ece9ad631fe635cf5e1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:06.233570 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a0071-666e-4ca2-9914-e026ed50b6ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:06.237258 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a0071-666e-4ca2-9914-e026ed50b6ef service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:06.253164 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b125d110-1a32-4a9d-9e78-072e931803b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f654defa4374ece9ad631fe635cf5e1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:06.254242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1338/2702] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:06.254985 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a0071-666e-4ca2-9914-e026ed50b6ef service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:06.256614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-3f8a0071-666e-4ca2-9914-e026ed50b6ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:06.274667 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:06.292758 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-b125d110-1a32-4a9d-9e78-072e931803b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f654defa4374ece9ad631fe635cf5e1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:06.305462 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1365/2703] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:06 2022] DELETE /identity/v3/users/9f654defa4374ece9ad631fe635cf5e1 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:06.311428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089d159-43ad-49b1-869f-9263fcb40781 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:06.311748 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089d159-43ad-49b1-869f-9263fcb40781 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:06.312048 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089d159-43ad-49b1-869f-9263fcb40781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:06.338059 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d089d159-43ad-49b1-869f-9263fcb40781 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:06.339357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d089d159-43ad-49b1-869f-9263fcb40781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:06.346733 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a0071-666e-4ca2-9914-e026ed50b6ef service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:06.417056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1366/2704] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:06.424546 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6969fa-4ead-4af8-8a4b-025a0ac5c856 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:06.424880 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6969fa-4ead-4af8-8a4b-025a0ac5c856 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:06.425350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6969fa-4ead-4af8-8a4b-025a0ac5c856 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:06.425680 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6969fa-4ead-4af8-8a4b-025a0ac5c856 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:06.427674 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-5f6969fa-4ead-4af8-8a4b-025a0ac5c856 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:06.440069 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089d159-43ad-49b1-869f-9263fcb40781 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:06.461608 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d089d159-43ad-49b1-869f-9263fcb40781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:06.500964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6969fa-4ead-4af8-8a4b-025a0ac5c856 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:06.502679 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6969fa-4ead-4af8-8a4b-025a0ac5c856 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:06.516363 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1339/2705] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:06 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:06.562301 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089d159-43ad-49b1-869f-9263fcb40781 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:06.571306 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d77144-b1a8-41e5-8911-c71a9c1515e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:06.572216 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d77144-b1a8-41e5-8911-c71a9c1515e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:06.572646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d77144-b1a8-41e5-8911-c71a9c1515e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:06.572936 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d77144-b1a8-41e5-8911-c71a9c1515e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:06.589714 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d77144-b1a8-41e5-8911-c71a9c1515e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:06.591447 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d77144-b1a8-41e5-8911-c71a9c1515e7 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:06.605022 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1367/2706] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:06.687659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6969fa-4ead-4af8-8a4b-025a0ac5c856 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:06.690569 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-5f6969fa-4ead-4af8-8a4b-025a0ac5c856 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:06.708748 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6969fa-4ead-4af8-8a4b-025a0ac5c856 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:06.720599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710a77d-f0a6-41ad-ba3d-e51714581179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:06.721645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710a77d-f0a6-41ad-ba3d-e51714581179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:06.722032 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710a77d-f0a6-41ad-ba3d-e51714581179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:06.722415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710a77d-f0a6-41ad-ba3d-e51714581179 admin admin] PATH_INFO: `/v3/users/2938858bcc7d445096cf884d107fdb9f` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:06.724613 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710a77d-f0a6-41ad-ba3d-e51714581179 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2938858bcc7d445096cf884d107fdb9f)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:06.726821 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710a77d-f0a6-41ad-ba3d-e51714581179 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:06.804959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:06.871187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7710a77d-f0a6-41ad-ba3d-e51714581179 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2938858bcc7d445096cf884d107fdb9f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:06.873035 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7710a77d-f0a6-41ad-ba3d-e51714581179 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2938858bcc7d445096cf884d107fdb9f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:06.879678 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7710a77d-f0a6-41ad-ba3d-e51714581179 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2938858bcc7d445096cf884d107fdb9f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:06.892191 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7710a77d-f0a6-41ad-ba3d-e51714581179 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2938858bcc7d445096cf884d107fdb9f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:06.904277 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7710a77d-f0a6-41ad-ba3d-e51714581179 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2938858bcc7d445096cf884d107fdb9f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:06.928772 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1340/2707] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:06 2022] DELETE /identity/v3/users/2938858bcc7d445096cf884d107fdb9f => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:06.933536 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054ed3b-fc0c-480c-9512-df2ea486c07b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:06.936527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054ed3b-fc0c-480c-9512-df2ea486c07b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:06.937160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054ed3b-fc0c-480c-9512-df2ea486c07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:06.963825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-d054ed3b-fc0c-480c-9512-df2ea486c07b None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:06.965095 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d054ed3b-fc0c-480c-9512-df2ea486c07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:07.046343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1368/2708] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:07.056363 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710a77d-f0a6-41ad-ba3d-e51714581179 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:07.058082 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-7710a77d-f0a6-41ad-ba3d-e51714581179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:07.126816 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710a77d-f0a6-41ad-ba3d-e51714581179 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:07.131480 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f49f78-9f1f-4cb4-9f49-c8d41670d50a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:07.132270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f49f78-9f1f-4cb4-9f49-c8d41670d50a service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:07.132555 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f49f78-9f1f-4cb4-9f49-c8d41670d50a service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:07.132825 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f49f78-9f1f-4cb4-9f49-c8d41670d50a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:07.145539 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f49f78-9f1f-4cb4-9f49-c8d41670d50a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:07.147291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f49f78-9f1f-4cb4-9f49-c8d41670d50a service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:07.159336 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1341/2709] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:07.593855 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054ed3b-fc0c-480c-9512-df2ea486c07b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:07.595358 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d054ed3b-fc0c-480c-9512-df2ea486c07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:07.609364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054ed3b-fc0c-480c-9512-df2ea486c07b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:07.618415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de56a8-2618-47e1-a452-bc11ab56470e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:07.619830 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de56a8-2618-47e1-a452-bc11ab56470e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:07.620358 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de56a8-2618-47e1-a452-bc11ab56470e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:07.620872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de56a8-2618-47e1-a452-bc11ab56470e admin admin] PATH_INFO: `/v3/projects/6a8bc5bd33e0432491df4938d4193db6` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:07.622545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de56a8-2618-47e1-a452-bc11ab56470e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a8bc5bd33e0432491df4938d4193db6)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:07.625243 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de56a8-2618-47e1-a452-bc11ab56470e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:07.653228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f49f78-9f1f-4cb4-9f49-c8d41670d50a service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:07.655283 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f4f49f78-9f1f-4cb4-9f49-c8d41670d50a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:07.668973 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:07.677143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f49f78-9f1f-4cb4-9f49-c8d41670d50a service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:07.689376 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f8de56a8-2618-47e1-a452-bc11ab56470e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a8bc5bd33e0432491df4938d4193db6', 'request_id': 'req-f8de56a8-2618-47e1-a452-bc11ab56470e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:07.695611 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f8de56a8-2618-47e1-a452-bc11ab56470e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a8bc5bd33e0432491df4938d4193db6', 'request_id': 'req-f8de56a8-2618-47e1-a452-bc11ab56470e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:07.706308 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f8de56a8-2618-47e1-a452-bc11ab56470e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a8bc5bd33e0432491df4938d4193db6', 'request_id': 'req-f8de56a8-2618-47e1-a452-bc11ab56470e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:07.730792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f8de56a8-2618-47e1-a452-bc11ab56470e admin admin] The token cache is being invalidate because project 6a8bc5bd33e0432491df4938d4193db6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:07.731077 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f8de56a8-2618-47e1-a452-bc11ab56470e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:07.734222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f8de56a8-2618-47e1-a452-bc11ab56470e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:07.735957 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1369/2710] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:07 2022] DELETE /identity/v3/projects/6a8bc5bd33e0432491df4938d4193db6 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:07.739956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de56a8-2618-47e1-a452-bc11ab56470e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:07.741426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f8de56a8-2618-47e1-a452-bc11ab56470e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:07.773356 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0f8d9-c26b-49a6-8890-80cadc0382fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:07.774106 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0f8d9-c26b-49a6-8890-80cadc0382fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:07.774460 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0f8d9-c26b-49a6-8890-80cadc0382fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:07.774803 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0f8d9-c26b-49a6-8890-80cadc0382fa admin admin] PATH_INFO: `/v3/projects/f263a05d46a34220ae5e529ac413e4a9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:07.776187 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0f8d9-c26b-49a6-8890-80cadc0382fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f263a05d46a34220ae5e529ac413e4a9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:07.780098 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0f8d9-c26b-49a6-8890-80cadc0382fa admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:07.840512 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de56a8-2618-47e1-a452-bc11ab56470e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:07.849174 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:07.859574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-68d0f8d9-c26b-49a6-8890-80cadc0382fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f263a05d46a34220ae5e529ac413e4a9', 'request_id': 'req-68d0f8d9-c26b-49a6-8890-80cadc0382fa'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:07.867715 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-68d0f8d9-c26b-49a6-8890-80cadc0382fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f263a05d46a34220ae5e529ac413e4a9', 'request_id': 'req-68d0f8d9-c26b-49a6-8890-80cadc0382fa'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:07.885364 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-68d0f8d9-c26b-49a6-8890-80cadc0382fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f263a05d46a34220ae5e529ac413e4a9', 'request_id': 'req-68d0f8d9-c26b-49a6-8890-80cadc0382fa'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:07.902385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-68d0f8d9-c26b-49a6-8890-80cadc0382fa admin admin] The token cache is being invalidate because project f263a05d46a34220ae5e529ac413e4a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:07.902829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-68d0f8d9-c26b-49a6-8890-80cadc0382fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:07.904337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-68d0f8d9-c26b-49a6-8890-80cadc0382fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:07.906550 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1342/2711] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:07 2022] DELETE /identity/v3/projects/f263a05d46a34220ae5e529ac413e4a9 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:07.907936 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe891a73-9e8d-4bb0-abe5-e868787cefaf None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:07.908459 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe891a73-9e8d-4bb0-abe5-e868787cefaf None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:07.908828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe891a73-9e8d-4bb0-abe5-e868787cefaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:07.933036 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822b50f-4761-46a1-8df0-188ca0167d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:07.934382 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822b50f-4761-46a1-8df0-188ca0167d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:07.935214 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822b50f-4761-46a1-8df0-188ca0167d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:07.935214 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822b50f-4761-46a1-8df0-188ca0167d24 admin admin] PATH_INFO: `/v3/users/37836f5859134685b1cab124c91f8a96` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:07.936993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822b50f-4761-46a1-8df0-188ca0167d24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37836f5859134685b1cab124c91f8a96)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:07.938707 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822b50f-4761-46a1-8df0-188ca0167d24 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:07.943561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-fe891a73-9e8d-4bb0-abe5-e868787cefaf None None] MFA Rules not processed for user `25fad82cbf944d7db13d6bf33d3297a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:07.944973 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fe891a73-9e8d-4bb0-abe5-e868787cefaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:08.001453 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8822b50f-4761-46a1-8df0-188ca0167d24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37836f5859134685b1cab124c91f8a96'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.002237 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8822b50f-4761-46a1-8df0-188ca0167d24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37836f5859134685b1cab124c91f8a96'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.010452 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8822b50f-4761-46a1-8df0-188ca0167d24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37836f5859134685b1cab124c91f8a96'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.020801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8822b50f-4761-46a1-8df0-188ca0167d24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37836f5859134685b1cab124c91f8a96'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.023466 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:08.027924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1343/2712] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:08.027924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe891a73-9e8d-4bb0-abe5-e868787cefaf None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:08.027924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fe891a73-9e8d-4bb0-abe5-e868787cefaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:08.043868 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe891a73-9e8d-4bb0-abe5-e868787cefaf None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:08.049109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-8822b50f-4761-46a1-8df0-188ca0167d24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37836f5859134685b1cab124c91f8a96'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.050242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6492783-0370-4394-9e96-1376c31c191a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:08.051102 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6492783-0370-4394-9e96-1376c31c191a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:08.051508 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6492783-0370-4394-9e96-1376c31c191a admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:08.051886 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6492783-0370-4394-9e96-1376c31c191a admin admin] PATH_INFO: `/v3/users/6dea7accf24c46e189aa92280ed77332` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:08.053124 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6492783-0370-4394-9e96-1376c31c191a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dea7accf24c46e189aa92280ed77332)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:08.055119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6492783-0370-4394-9e96-1376c31c191a admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:08.059374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1370/2713] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:07 2022] DELETE /identity/v3/users/37836f5859134685b1cab124c91f8a96 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:08.060935 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822b50f-4761-46a1-8df0-188ca0167d24 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:08.063175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-8822b50f-4761-46a1-8df0-188ca0167d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:08.148581 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a6492783-0370-4394-9e96-1376c31c191a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dea7accf24c46e189aa92280ed77332'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.150297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a6492783-0370-4394-9e96-1376c31c191a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dea7accf24c46e189aa92280ed77332'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.151032 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a6492783-0370-4394-9e96-1376c31c191a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dea7accf24c46e189aa92280ed77332'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.162858 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822b50f-4761-46a1-8df0-188ca0167d24 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:08.165809 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a6492783-0370-4394-9e96-1376c31c191a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dea7accf24c46e189aa92280ed77332'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.168011 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4bd65-d12c-4731-bf7b-d9bf096f51a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:08.169550 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4bd65-d12c-4731-bf7b-d9bf096f51a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:08.169847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4bd65-d12c-4731-bf7b-d9bf096f51a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:08.170433 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4bd65-d12c-4731-bf7b-d9bf096f51a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:08.173058 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-16d4bd65-d12c-4731-bf7b-d9bf096f51a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:08.179094 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a6492783-0370-4394-9e96-1376c31c191a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dea7accf24c46e189aa92280ed77332'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.218010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1344/2714] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:08 2022] DELETE /identity/v3/users/6dea7accf24c46e189aa92280ed77332 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:08.274837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4bd65-d12c-4731-bf7b-d9bf096f51a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:08.276295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4bd65-d12c-4731-bf7b-d9bf096f51a3 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:08.289074 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1371/2715] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:08.427013 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6492783-0370-4394-9e96-1376c31c191a admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:08.428829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a6492783-0370-4394-9e96-1376c31c191a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:08.503165 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:08.553503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6492783-0370-4394-9e96-1376c31c191a admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:08.561230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17e094-2110-49da-925c-59735d1a0007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:08.561958 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17e094-2110-49da-925c-59735d1a0007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:08.562444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17e094-2110-49da-925c-59735d1a0007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:08.562778 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17e094-2110-49da-925c-59735d1a0007 admin admin] PATH_INFO: `/v3/projects/9444d921e63940a2aac749d526f61bd4` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:08.564147 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17e094-2110-49da-925c-59735d1a0007 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9444d921e63940a2aac749d526f61bd4)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:08.567146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17e094-2110-49da-925c-59735d1a0007 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:08.650655 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2f17e094-2110-49da-925c-59735d1a0007 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9444d921e63940a2aac749d526f61bd4', 'request_id': 'req-2f17e094-2110-49da-925c-59735d1a0007'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.666517 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2f17e094-2110-49da-925c-59735d1a0007 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9444d921e63940a2aac749d526f61bd4', 'request_id': 'req-2f17e094-2110-49da-925c-59735d1a0007'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.687833 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2f17e094-2110-49da-925c-59735d1a0007 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9444d921e63940a2aac749d526f61bd4', 'request_id': 'req-2f17e094-2110-49da-925c-59735d1a0007'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.704694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2f17e094-2110-49da-925c-59735d1a0007 admin admin] The token cache is being invalidate because project 9444d921e63940a2aac749d526f61bd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:08.706143 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2f17e094-2110-49da-925c-59735d1a0007 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.707550 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-2f17e094-2110-49da-925c-59735d1a0007 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.710891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1345/2716] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:08 2022] DELETE /identity/v3/projects/9444d921e63940a2aac749d526f61bd4 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:08.718393 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212c4a3-977f-4014-8a7b-7f0750765457 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:08.718866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212c4a3-977f-4014-8a7b-7f0750765457 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:08.719320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212c4a3-977f-4014-8a7b-7f0750765457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:08.755388 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-b212c4a3-977f-4014-8a7b-7f0750765457 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:08.757239 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b212c4a3-977f-4014-8a7b-7f0750765457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:08.810447 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17e094-2110-49da-925c-59735d1a0007 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:08.813556 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-2f17e094-2110-49da-925c-59735d1a0007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:08.861075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1372/2717] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:08.869209 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a33e4-32ac-4310-8b44-3b622f8e48cc None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:08.869571 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a33e4-32ac-4310-8b44-3b622f8e48cc None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:08.869939 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a33e4-32ac-4310-8b44-3b622f8e48cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:08.903613 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-a65a33e4-32ac-4310-8b44-3b622f8e48cc None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:08.905143 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a65a33e4-32ac-4310-8b44-3b622f8e48cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:08.909448 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17e094-2110-49da-925c-59735d1a0007 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:08.915607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65b117-38b4-4ae8-abf1-f4028dcfe31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:08.916281 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65b117-38b4-4ae8-abf1-f4028dcfe31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:08.916632 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65b117-38b4-4ae8-abf1-f4028dcfe31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:08.916738 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1373/2718] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:08.916910 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65b117-38b4-4ae8-abf1-f4028dcfe31f admin admin] PATH_INFO: `/v3/projects/965cf87102104082bf1be975d88cb9f3` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:08.918162 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65b117-38b4-4ae8-abf1-f4028dcfe31f admin admin] RBAC: Authorizing `identity:delete_project(project_id=965cf87102104082bf1be975d88cb9f3)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:08.918821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a33e4-32ac-4310-8b44-3b622f8e48cc None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:08.920666 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65b117-38b4-4ae8-abf1-f4028dcfe31f admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:08.929557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a33e4-32ac-4310-8b44-3b622f8e48cc None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:08.935861 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb430a9-b552-4fc3-b802-2a93855200e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:08.936487 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb430a9-b552-4fc3-b802-2a93855200e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:08.936749 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb430a9-b552-4fc3-b802-2a93855200e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:08.937022 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb430a9-b552-4fc3-b802-2a93855200e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:08.937677 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb430a9-b552-4fc3-b802-2a93855200e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:08.938123 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb430a9-b552-4fc3-b802-2a93855200e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:08.939790 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb430a9-b552-4fc3-b802-2a93855200e7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:08.953918 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1374/2719] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:40:08.958307 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb430a9-b552-4fc3-b802-2a93855200e7 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:08.970169 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb430a9-b552-4fc3-b802-2a93855200e7 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:08.974099 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7d65b117-38b4-4ae8-abf1-f4028dcfe31f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '965cf87102104082bf1be975d88cb9f3', 'request_id': 'req-7d65b117-38b4-4ae8-abf1-f4028dcfe31f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.982402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7d65b117-38b4-4ae8-abf1-f4028dcfe31f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '965cf87102104082bf1be975d88cb9f3', 'request_id': 'req-7d65b117-38b4-4ae8-abf1-f4028dcfe31f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:08.985057 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:08.986729 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:08.992227 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7d65b117-38b4-4ae8-abf1-f4028dcfe31f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '965cf87102104082bf1be975d88cb9f3', 'request_id': 'req-7d65b117-38b4-4ae8-abf1-f4028dcfe31f'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:09.009861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7d65b117-38b4-4ae8-abf1-f4028dcfe31f admin admin] The token cache is being invalidate because project 965cf87102104082bf1be975d88cb9f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:09.010164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7d65b117-38b4-4ae8-abf1-f4028dcfe31f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:09.011265 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-7d65b117-38b4-4ae8-abf1-f4028dcfe31f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:09.013341 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1346/2720] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:08 2022] DELETE /identity/v3/projects/965cf87102104082bf1be975d88cb9f3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:09.018239 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a411231-f554-4e4d-b1a7-4e90cab6fbef None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.018239 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a411231-f554-4e4d-b1a7-4e90cab6fbef None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.018490 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a411231-f554-4e4d-b1a7-4e90cab6fbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.041672 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-5a411231-f554-4e4d-b1a7-4e90cab6fbef None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:09.042808 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-5a411231-f554-4e4d-b1a7-4e90cab6fbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:09.043371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e9ce8-b97e-4613-9725-5d20695c722c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:09.044595 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e9ce8-b97e-4613-9725-5d20695c722c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.044990 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e9ce8-b97e-4613-9725-5d20695c722c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.045325 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e9ce8-b97e-4613-9725-5d20695c722c admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.045834 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e9ce8-b97e-4613-9725-5d20695c722c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:09.048434 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e9ce8-b97e-4613-9725-5d20695c722c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:09.051648 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1347/2721] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:09.061207 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900c01c-85c6-40ab-8977-ce59dd3c9e19 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.061495 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900c01c-85c6-40ab-8977-ce59dd3c9e19 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.061777 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900c01c-85c6-40ab-8977-ce59dd3c9e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.082742 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1375/2722] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:08 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:09.083118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-1900c01c-85c6-40ab-8977-ce59dd3c9e19 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:09.083722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e9ce8-b97e-4613-9725-5d20695c722c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:09.084973 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1900c01c-85c6-40ab-8977-ce59dd3c9e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:09.085350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f33e9ce8-b97e-4613-9725-5d20695c722c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:09.149145 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1348/2723] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:09.153395 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900c01c-85c6-40ab-8977-ce59dd3c9e19 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:09.154156 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e9ce8-b97e-4613-9725-5d20695c722c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:09.158874 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58e13d-9336-4a05-8d23-c977915e5ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:09.159482 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58e13d-9336-4a05-8d23-c977915e5ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.159813 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58e13d-9336-4a05-8d23-c977915e5ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.160071 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58e13d-9336-4a05-8d23-c977915e5ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.160696 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e58e13d-9336-4a05-8d23-c977915e5ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:09.162651 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e58e13d-9336-4a05-8d23-c977915e5ec7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:09.162766 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900c01c-85c6-40ab-8977-ce59dd3c9e19 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:09.167754 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96c720-2b8e-4f01-8fe7-2d75f55cfa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:09.168511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96c720-2b8e-4f01-8fe7-2d75f55cfa40 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.168791 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96c720-2b8e-4f01-8fe7-2d75f55cfa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.169233 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96c720-2b8e-4f01-8fe7-2d75f55cfa40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.170183 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96c720-2b8e-4f01-8fe7-2d75f55cfa40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:09.170534 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96c720-2b8e-4f01-8fe7-2d75f55cfa40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:09.172345 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc96c720-2b8e-4f01-8fe7-2d75f55cfa40 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:09.186056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1376/2724] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:09 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:09.187047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1349/2725] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:40:09.189934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58e13d-9336-4a05-8d23-c977915e5ec7 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:09.192970 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96c720-2b8e-4f01-8fe7-2d75f55cfa40 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:09.201796 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58e13d-9336-4a05-8d23-c977915e5ec7 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:09.203919 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96c720-2b8e-4f01-8fe7-2d75f55cfa40 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:09.206598 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f7a8d-b7e5-4bee-bb06-e3b60b22df36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:09.207251 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f7a8d-b7e5-4bee-bb06-e3b60b22df36 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.207530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f7a8d-b7e5-4bee-bb06-e3b60b22df36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.207837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f7a8d-b7e5-4bee-bb06-e3b60b22df36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.208489 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f7a8d-b7e5-4bee-bb06-e3b60b22df36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:09.208590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909209d-02dc-4e5f-9799-ab0b7b058ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:09.209102 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909209d-02dc-4e5f-9799-ab0b7b058ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.209384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909209d-02dc-4e5f-9799-ab0b7b058ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.209661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909209d-02dc-4e5f-9799-ab0b7b058ff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.210082 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f7a8d-b7e5-4bee-bb06-e3b60b22df36 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:09.210281 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1909209d-02dc-4e5f-9799-ab0b7b058ff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:09.212218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1909209d-02dc-4e5f-9799-ab0b7b058ff4 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:09.220656 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1377/2726] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:09.225184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f7a8d-b7e5-4bee-bb06-e3b60b22df36 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:09.236526 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f7a8d-b7e5-4bee-bb06-e3b60b22df36 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:09.242779 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1350/2727] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:09 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:09.242917 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99764b7-5f85-4d4c-a5c5-02621f9e7873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:09.242917 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99764b7-5f85-4d4c-a5c5-02621f9e7873 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.242917 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99764b7-5f85-4d4c-a5c5-02621f9e7873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.242917 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99764b7-5f85-4d4c-a5c5-02621f9e7873 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.243302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99764b7-5f85-4d4c-a5c5-02621f9e7873 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:09.243838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909209d-02dc-4e5f-9799-ab0b7b058ff4 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:09.245045 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99764b7-5f85-4d4c-a5c5-02621f9e7873 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:09.259826 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1378/2728] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:09.264603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99764b7-5f85-4d4c-a5c5-02621f9e7873 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:09.275833 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99764b7-5f85-4d4c-a5c5-02621f9e7873 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:09.310359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909209d-02dc-4e5f-9799-ab0b7b058ff4 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:09.315891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df1c03-740d-46c0-bc26-979fa146497b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:09.316847 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df1c03-740d-46c0-bc26-979fa146497b admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.317315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df1c03-740d-46c0-bc26-979fa146497b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.317799 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df1c03-740d-46c0-bc26-979fa146497b admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.318916 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df1c03-740d-46c0-bc26-979fa146497b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:09.322040 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df1c03-740d-46c0-bc26-979fa146497b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:09.355206 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578967ac-41ac-4c33-9206-2672dd2cf70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:09.359103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578967ac-41ac-4c33-9206-2672dd2cf70e admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.359242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1351/2729] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:09 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:09.359367 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578967ac-41ac-4c33-9206-2672dd2cf70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.359367 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578967ac-41ac-4c33-9206-2672dd2cf70e admin admin] PATH_INFO: `/v3/projects/1a214a1227ae426ea4270b54904c7880/users/c9c6d6ceb012410ca6e6b1353fc3f22f/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.360169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df1c03-740d-46c0-bc26-979fa146497b admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:09.369879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:09.370510 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df1c03-740d-46c0-bc26-979fa146497b admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:09.375580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578967ac-41ac-4c33-9206-2672dd2cf70e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a214a1227ae426ea4270b54904c7880, user_id=c9c6d6ceb012410ca6e6b1353fc3f22f, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:09.378274 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578967ac-41ac-4c33-9206-2672dd2cf70e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:09.378768 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f9f1a1-826a-4e5f-8463-d8a9e8f45aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:09.379385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9f1a1-826a-4e5f-8463-d8a9e8f45aac admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.379726 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9f1a1-826a-4e5f-8463-d8a9e8f45aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.380005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9f1a1-826a-4e5f-8463-d8a9e8f45aac admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.380661 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f9f1a1-826a-4e5f-8463-d8a9e8f45aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:09.382289 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f9f1a1-826a-4e5f-8463-d8a9e8f45aac admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:09.393584 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1352/2730] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:09.396239 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1379/2731] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:09 2022] PUT /identity/v3/projects/1a214a1227ae426ea4270b54904c7880/users/c9c6d6ceb012410ca6e6b1353fc3f22f/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:09.400568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f9f1a1-826a-4e5f-8463-d8a9e8f45aac admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:09.411316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275aba2-b502-4ed0-aadf-45bec9cc2754 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.411707 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275aba2-b502-4ed0-aadf-45bec9cc2754 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.412265 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275aba2-b502-4ed0-aadf-45bec9cc2754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.415964 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f9f1a1-826a-4e5f-8463-d8a9e8f45aac admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:09.442857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-2275aba2-b502-4ed0-aadf-45bec9cc2754 None None] MFA Rules not processed for user `c9c6d6ceb012410ca6e6b1353fc3f22f`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:09.444238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2275aba2-b502-4ed0-aadf-45bec9cc2754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:09.445959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:09.491368 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d23408-78c8-4ab0-8114-b6263a4614eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:09.492031 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d23408-78c8-4ab0-8114-b6263a4614eb admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.492308 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d23408-78c8-4ab0-8114-b6263a4614eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.492580 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d23408-78c8-4ab0-8114-b6263a4614eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.493244 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d23408-78c8-4ab0-8114-b6263a4614eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:09.494831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d23408-78c8-4ab0-8114-b6263a4614eb admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:09.505318 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1353/2732] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:09.506315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf21de-f52c-4fda-92a0-fc7d0259f869 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.506594 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf21de-f52c-4fda-92a0-fc7d0259f869 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.506866 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bf21de-f52c-4fda-92a0-fc7d0259f869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.514730 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1380/2733] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:09.515392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275aba2-b502-4ed0-aadf-45bec9cc2754 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:09.526795 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275aba2-b502-4ed0-aadf-45bec9cc2754 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:09.531561 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-c1bf21de-f52c-4fda-92a0-fc7d0259f869 None None] MFA Rules not processed for user `18f8f419b523440f8a4ca06dc83bff40`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:09.532459 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38198391-ea80-4825-adf1-53cb6bd3f4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:09.532861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-c1bf21de-f52c-4fda-92a0-fc7d0259f869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:09.533116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38198391-ea80-4825-adf1-53cb6bd3f4e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.533401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38198391-ea80-4825-adf1-53cb6bd3f4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.533682 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38198391-ea80-4825-adf1-53cb6bd3f4e5 admin admin] PATH_INFO: `/v3/projects/0fe613ad4ba840a9b29cc41530b51a0d/users/38a83dd8096d4d1392f55a9a12248909/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.549296 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38198391-ea80-4825-adf1-53cb6bd3f4e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe613ad4ba840a9b29cc41530b51a0d, user_id=38a83dd8096d4d1392f55a9a12248909, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:09.552000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38198391-ea80-4825-adf1-53cb6bd3f4e5 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:09.567386 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1381/2734] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:09 2022] PUT /identity/v3/projects/0fe613ad4ba840a9b29cc41530b51a0d/users/38a83dd8096d4d1392f55a9a12248909/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:09.572727 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38198391-ea80-4825-adf1-53cb6bd3f4e5 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:09.594668 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1354/2735] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:09.596045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234e2a3-950c-44c9-9d2c-eaa512f3b4aa None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.596404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234e2a3-950c-44c9-9d2c-eaa512f3b4aa None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.596687 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234e2a3-950c-44c9-9d2c-eaa512f3b4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.630423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-6234e2a3-950c-44c9-9d2c-eaa512f3b4aa None None] MFA Rules not processed for user `38a83dd8096d4d1392f55a9a12248909`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:09.631815 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-6234e2a3-950c-44c9-9d2c-eaa512f3b4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:09.646101 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38198391-ea80-4825-adf1-53cb6bd3f4e5 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:09.651327 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98ff6c-467a-4027-b3cf-ce9d6d67b475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:09.651966 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98ff6c-467a-4027-b3cf-ce9d6d67b475 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.652343 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98ff6c-467a-4027-b3cf-ce9d6d67b475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.653238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98ff6c-467a-4027-b3cf-ce9d6d67b475 admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.653490 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98ff6c-467a-4027-b3cf-ce9d6d67b475 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:09.655807 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98ff6c-467a-4027-b3cf-ce9d6d67b475 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:09.689708 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1382/2736] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:09 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:09.693380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98ff6c-467a-4027-b3cf-ce9d6d67b475 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:09.708129 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1355/2737] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:09.712407 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234e2a3-950c-44c9-9d2c-eaa512f3b4aa None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:09.725095 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234e2a3-950c-44c9-9d2c-eaa512f3b4aa None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:09.755312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:09.777821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98ff6c-467a-4027-b3cf-ce9d6d67b475 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:09.785854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620ba79-e151-4e83-85ea-90e13d68837a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:09.786619 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620ba79-e151-4e83-85ea-90e13d68837a admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.787003 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620ba79-e151-4e83-85ea-90e13d68837a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.787496 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620ba79-e151-4e83-85ea-90e13d68837a admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.788360 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620ba79-e151-4e83-85ea-90e13d68837a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:09.790428 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620ba79-e151-4e83-85ea-90e13d68837a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:09.809033 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03522577-f94a-4a5c-815b-ff53dc3f08af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:09.810817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03522577-f94a-4a5c-815b-ff53dc3f08af admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.811941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03522577-f94a-4a5c-815b-ff53dc3f08af admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.812376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03522577-f94a-4a5c-815b-ff53dc3f08af admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.813164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03522577-f94a-4a5c-815b-ff53dc3f08af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:09.816008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03522577-f94a-4a5c-815b-ff53dc3f08af admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:09.819312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1383/2738] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:09 2022] POST /identity/v3/users => generated 299 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:09.819312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1882c3-d580-4668-89dd-372df23690d1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.819312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1882c3-d580-4668-89dd-372df23690d1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.819312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1882c3-d580-4668-89dd-372df23690d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.821394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:09.840415 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-cf1882c3-d580-4668-89dd-372df23690d1 None None] MFA Rules not processed for user `25fad82cbf944d7db13d6bf33d3297a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:09.843896 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cf1882c3-d580-4668-89dd-372df23690d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:09.849321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1356/2739] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:09 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:09.850154 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03522577-f94a-4a5c-815b-ff53dc3f08af admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:09.925441 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1384/2740] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:09.925441 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1882c3-d580-4668-89dd-372df23690d1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:09.935659 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1882c3-d580-4668-89dd-372df23690d1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:09.949843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03522577-f94a-4a5c-815b-ff53dc3f08af admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:09.955078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06446b-4c8c-4079-8f9a-0ce7860a4e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:09.955876 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06446b-4c8c-4079-8f9a-0ce7860a4e6d admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:09.956362 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06446b-4c8c-4079-8f9a-0ce7860a4e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:09.956700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06446b-4c8c-4079-8f9a-0ce7860a4e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:09.957453 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06446b-4c8c-4079-8f9a-0ce7860a4e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:09.959131 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06446b-4c8c-4079-8f9a-0ce7860a4e6d admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:09.972297 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1357/2741] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:09 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:09.976942 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06446b-4c8c-4079-8f9a-0ce7860a4e6d admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:09.992201 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06446b-4c8c-4079-8f9a-0ce7860a4e6d admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:10.000826 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9c1ea-f8ac-448f-a269-1bba02f8bd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:10.001629 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9c1ea-f8ac-448f-a269-1bba02f8bd2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:10.001975 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9c1ea-f8ac-448f-a269-1bba02f8bd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:10.002329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9c1ea-f8ac-448f-a269-1bba02f8bd2c admin admin] PATH_INFO: `/v3/projects/6d33e517a4cc4e2c9bbb3387cd81f19f/users/72fceb59be78425d8fceefecfce1e254/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:10.011396 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6648a5e-1d8a-453e-9c09-388a492e21be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:10.012147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6648a5e-1d8a-453e-9c09-388a492e21be admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:10.012511 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6648a5e-1d8a-453e-9c09-388a492e21be admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:10.013018 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6648a5e-1d8a-453e-9c09-388a492e21be admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:10.013762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6648a5e-1d8a-453e-9c09-388a492e21be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:10.015874 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6648a5e-1d8a-453e-9c09-388a492e21be admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:10.021955 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9c1ea-f8ac-448f-a269-1bba02f8bd2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d33e517a4cc4e2c9bbb3387cd81f19f, user_id=72fceb59be78425d8fceefecfce1e254, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:10.025238 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9c1ea-f8ac-448f-a269-1bba02f8bd2c admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:10.046382 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1385/2742] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:09 2022] POST /identity/v3/users => generated 298 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:10.047416 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6648a5e-1d8a-453e-9c09-388a492e21be admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:10.060209 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6648a5e-1d8a-453e-9c09-388a492e21be admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:10.065548 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1358/2743] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:09 2022] PUT /identity/v3/projects/6d33e517a4cc4e2c9bbb3387cd81f19f/users/72fceb59be78425d8fceefecfce1e254/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:10.070250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9c1ea-f8ac-448f-a269-1bba02f8bd2c admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:10.170061 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb4648-cdb5-4bd6-8202-97095ca81696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:10.173562 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb4648-cdb5-4bd6-8202-97095ca81696 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:10.174063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9c1ea-f8ac-448f-a269-1bba02f8bd2c admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:10.174541 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb4648-cdb5-4bd6-8202-97095ca81696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:10.175545 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb4648-cdb5-4bd6-8202-97095ca81696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:10.176346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb4648-cdb5-4bd6-8202-97095ca81696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:10.178936 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb4648-cdb5-4bd6-8202-97095ca81696 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:10.188060 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce638ae-69aa-4d8e-82a2-2a1997f5ff88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:10.188872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce638ae-69aa-4d8e-82a2-2a1997f5ff88 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:10.189293 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce638ae-69aa-4d8e-82a2-2a1997f5ff88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:10.189658 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce638ae-69aa-4d8e-82a2-2a1997f5ff88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:10.190400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce638ae-69aa-4d8e-82a2-2a1997f5ff88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:10.192228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce638ae-69aa-4d8e-82a2-2a1997f5ff88 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:10.204339 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1386/2744] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:10.210880 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb4648-cdb5-4bd6-8202-97095ca81696 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:10.214607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1359/2745] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:10.215806 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce638ae-69aa-4d8e-82a2-2a1997f5ff88 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:10.221600 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:10.228431 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce638ae-69aa-4d8e-82a2-2a1997f5ff88 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:10.229446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb4648-cdb5-4bd6-8202-97095ca81696 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:10.234642 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2b755-daa2-45bd-afcf-ac5305e7fa64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:10.236144 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2b755-daa2-45bd-afcf-ac5305e7fa64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:10.237330 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2b755-daa2-45bd-afcf-ac5305e7fa64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:10.237330 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2b755-daa2-45bd-afcf-ac5305e7fa64 admin admin] PATH_INFO: `/v3/projects/6d33e517a4cc4e2c9bbb3387cd81f19f/users/72fceb59be78425d8fceefecfce1e254/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:10.237850 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e04ab-6547-4039-9908-e2b68df426fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:10.238870 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e04ab-6547-4039-9908-e2b68df426fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:10.239684 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e04ab-6547-4039-9908-e2b68df426fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:10.240157 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e04ab-6547-4039-9908-e2b68df426fb admin admin] PATH_INFO: `/v3/projects/06f034d318f04d05a3112f979dcb754a/users/6efb7eea519e44e098088f0dd54e1f96/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:10.240667 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2b755-daa2-45bd-afcf-ac5305e7fa64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d33e517a4cc4e2c9bbb3387cd81f19f, user_id=72fceb59be78425d8fceefecfce1e254, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:10.244138 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2b755-daa2-45bd-afcf-ac5305e7fa64 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:10.268359 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1360/2746] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:10 2022] PUT /identity/v3/projects/6d33e517a4cc4e2c9bbb3387cd81f19f/users/72fceb59be78425d8fceefecfce1e254/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:10.268807 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e04ab-6547-4039-9908-e2b68df426fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=06f034d318f04d05a3112f979dcb754a, user_id=6efb7eea519e44e098088f0dd54e1f96, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:10.268807 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e04ab-6547-4039-9908-e2b68df426fb admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:10.271479 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64a6f0-a7c5-4945-8f9d-3ea776ee7db8 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:10.271804 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64a6f0-a7c5-4945-8f9d-3ea776ee7db8 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:10.272087 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64a6f0-a7c5-4945-8f9d-3ea776ee7db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:10.289947 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1387/2747] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:10 2022] PUT /identity/v3/projects/06f034d318f04d05a3112f979dcb754a/users/6efb7eea519e44e098088f0dd54e1f96/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:10.293855 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e04ab-6547-4039-9908-e2b68df426fb admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:10.310074 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-fc64a6f0-a7c5-4945-8f9d-3ea776ee7db8 None None] MFA Rules not processed for user `72fceb59be78425d8fceefecfce1e254`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:10.311262 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fc64a6f0-a7c5-4945-8f9d-3ea776ee7db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:10.362146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:10.377446 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e04ab-6547-4039-9908-e2b68df426fb admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:10.382920 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33133b3-85a2-4cf5-a513-287717b5a1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:10.383648 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33133b3-85a2-4cf5-a513-287717b5a1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:10.383993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33133b3-85a2-4cf5-a513-287717b5a1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:10.384120 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33133b3-85a2-4cf5-a513-287717b5a1e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:10.384738 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33133b3-85a2-4cf5-a513-287717b5a1e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:10.394596 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33133b3-85a2-4cf5-a513-287717b5a1e9 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:10.419056 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1388/2748] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:10 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:10.424319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33133b3-85a2-4cf5-a513-287717b5a1e9 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:10.429212 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1361/2749] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:10.441163 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33133b3-85a2-4cf5-a513-287717b5a1e9 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:10.446583 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64a6f0-a7c5-4945-8f9d-3ea776ee7db8 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:10.447844 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f0894-d31c-415e-bedb-f2be55ccd9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:10.448395 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-fc64a6f0-a7c5-4945-8f9d-3ea776ee7db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:10.448868 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f0894-d31c-415e-bedb-f2be55ccd9c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:10.449561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f0894-d31c-415e-bedb-f2be55ccd9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:10.449984 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f0894-d31c-415e-bedb-f2be55ccd9c7 admin admin] PATH_INFO: `/v3/projects/06f034d318f04d05a3112f979dcb754a/users/6efb7eea519e44e098088f0dd54e1f96/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:10.453052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f0894-d31c-415e-bedb-f2be55ccd9c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06f034d318f04d05a3112f979dcb754a, user_id=6efb7eea519e44e098088f0dd54e1f96, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:10.456358 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f0894-d31c-415e-bedb-f2be55ccd9c7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:10.477525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1389/2750] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:10 2022] PUT /identity/v3/projects/06f034d318f04d05a3112f979dcb754a/users/6efb7eea519e44e098088f0dd54e1f96/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:10.485232 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f0894-d31c-415e-bedb-f2be55ccd9c7 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:10.488321 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-a73f0894-d31c-415e-bedb-f2be55ccd9c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:10.533405 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64a6f0-a7c5-4945-8f9d-3ea776ee7db8 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:10.580161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f0894-d31c-415e-bedb-f2be55ccd9c7 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:10.584290 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8535d54-6e9d-4d90-94f0-53c0e497169a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:10.585282 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8535d54-6e9d-4d90-94f0-53c0e497169a service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:10.585851 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8535d54-6e9d-4d90-94f0-53c0e497169a service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:10.586281 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8535d54-6e9d-4d90-94f0-53c0e497169a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:10.588491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-d8535d54-6e9d-4d90-94f0-53c0e497169a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:10.601595 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8535d54-6e9d-4d90-94f0-53c0e497169a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:10.602843 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da455c-78f3-44a3-9c68-67f1385631d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:10.603516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da455c-78f3-44a3-9c68-67f1385631d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:10.603834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da455c-78f3-44a3-9c68-67f1385631d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:10.604041 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8535d54-6e9d-4d90-94f0-53c0e497169a service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:10.604198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da455c-78f3-44a3-9c68-67f1385631d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:10.615136 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1390/2751] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:10.616489 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aeeb3e-72da-4b4f-8ea4-279fb6f9357a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:10.616879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aeeb3e-72da-4b4f-8ea4-279fb6f9357a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:10.617160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aeeb3e-72da-4b4f-8ea4-279fb6f9357a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:10.658271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-36aeeb3e-72da-4b4f-8ea4-279fb6f9357a None None] MFA Rules not processed for user `6efb7eea519e44e098088f0dd54e1f96`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:10.660243 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-36aeeb3e-72da-4b4f-8ea4-279fb6f9357a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:10.679572 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da455c-78f3-44a3-9c68-67f1385631d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:10.681521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62da455c-78f3-44a3-9c68-67f1385631d7 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:10.706501 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1362/2752] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:10.735912 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:10.772706 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1391/2753] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:10.789676 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da455c-78f3-44a3-9c68-67f1385631d7 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:10.795345 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-62da455c-78f3-44a3-9c68-67f1385631d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:10.822386 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da455c-78f3-44a3-9c68-67f1385631d7 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:10.829942 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f4b30-a056-453a-9390-586324b6e271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:10.830566 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f4b30-a056-453a-9390-586324b6e271 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:10.830844 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f4b30-a056-453a-9390-586324b6e271 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:10.831113 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f4b30-a056-453a-9390-586324b6e271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:10.850822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f4b30-a056-453a-9390-586324b6e271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:10.854516 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f4b30-a056-453a-9390-586324b6e271 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:10.899008 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1363/2754] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:10.978571 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aeeb3e-72da-4b4f-8ea4-279fb6f9357a None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:10.999575 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aeeb3e-72da-4b4f-8ea4-279fb6f9357a None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:11.005925 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe43f55-0bad-4250-a946-21b3954ae0f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:11.007239 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe43f55-0bad-4250-a946-21b3954ae0f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:11.007239 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe43f55-0bad-4250-a946-21b3954ae0f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:11.007522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe43f55-0bad-4250-a946-21b3954ae0f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:11.088059 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe43f55-0bad-4250-a946-21b3954ae0f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:11.089406 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe43f55-0bad-4250-a946-21b3954ae0f4 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:11.104211 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1392/2755] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:11.608302 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f4b30-a056-453a-9390-586324b6e271 service cinder] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:11.636846 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:11.677570 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f4b30-a056-453a-9390-586324b6e271 service cinder] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:11.682234 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0617578-02f2-41f2-8afe-fef16e653198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:11.682862 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0617578-02f2-41f2-8afe-fef16e653198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:11.683140 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0617578-02f2-41f2-8afe-fef16e653198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:11.683408 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0617578-02f2-41f2-8afe-fef16e653198 admin admin] PATH_INFO: `/v3/users/38a83dd8096d4d1392f55a9a12248909` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:11.684738 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0617578-02f2-41f2-8afe-fef16e653198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38a83dd8096d4d1392f55a9a12248909)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:11.686788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0617578-02f2-41f2-8afe-fef16e653198 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:11.733181 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a0617578-02f2-41f2-8afe-fef16e653198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38a83dd8096d4d1392f55a9a12248909'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:11.733836 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a0617578-02f2-41f2-8afe-fef16e653198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38a83dd8096d4d1392f55a9a12248909'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:11.734441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a0617578-02f2-41f2-8afe-fef16e653198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38a83dd8096d4d1392f55a9a12248909'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:11.741351 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a0617578-02f2-41f2-8afe-fef16e653198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38a83dd8096d4d1392f55a9a12248909'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:11.748514 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-a0617578-02f2-41f2-8afe-fef16e653198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38a83dd8096d4d1392f55a9a12248909'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:11.766428 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1364/2756] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:11 2022] DELETE /identity/v3/users/38a83dd8096d4d1392f55a9a12248909 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:11.771053 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fe284-0455-48c0-a8b4-9b522e237ba1 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:11.771353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fe284-0455-48c0-a8b4-9b522e237ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:11.771642 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fe284-0455-48c0-a8b4-9b522e237ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:11.791865 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-477fe284-0455-48c0-a8b4-9b522e237ba1 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:11.793079 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-477fe284-0455-48c0-a8b4-9b522e237ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:11.856122 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1393/2757] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:11.867760 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0617578-02f2-41f2-8afe-fef16e653198 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:11.869902 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a0617578-02f2-41f2-8afe-fef16e653198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:11.988872 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0617578-02f2-41f2-8afe-fef16e653198 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:12.000756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0253d-0c4a-4146-8e1d-95ac116365e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:12.001618 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0253d-0c4a-4146-8e1d-95ac116365e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:12.002093 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0253d-0c4a-4146-8e1d-95ac116365e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:12.002523 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0253d-0c4a-4146-8e1d-95ac116365e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:12.014805 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0253d-0c4a-4146-8e1d-95ac116365e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:12.016596 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c0253d-0c4a-4146-8e1d-95ac116365e0 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:12.027681 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1365/2758] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:12.412695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477fe284-0455-48c0-a8b4-9b522e237ba1 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:12.414838 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-477fe284-0455-48c0-a8b4-9b522e237ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:12.456160 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477fe284-0455-48c0-a8b4-9b522e237ba1 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:12.461991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930d2ed-bb0c-4852-b931-562e2e330580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:12.462882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930d2ed-bb0c-4852-b931-562e2e330580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:12.464117 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930d2ed-bb0c-4852-b931-562e2e330580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:12.464511 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930d2ed-bb0c-4852-b931-562e2e330580 admin admin] PATH_INFO: `/v3/projects/0fe613ad4ba840a9b29cc41530b51a0d` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:12.466065 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930d2ed-bb0c-4852-b931-562e2e330580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe613ad4ba840a9b29cc41530b51a0d)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:12.468606 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930d2ed-bb0c-4852-b931-562e2e330580 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:12.573620 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1930d2ed-bb0c-4852-b931-562e2e330580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe613ad4ba840a9b29cc41530b51a0d', 'request_id': 'req-1930d2ed-bb0c-4852-b931-562e2e330580'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:12.595363 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1930d2ed-bb0c-4852-b931-562e2e330580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe613ad4ba840a9b29cc41530b51a0d', 'request_id': 'req-1930d2ed-bb0c-4852-b931-562e2e330580'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:12.615324 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1930d2ed-bb0c-4852-b931-562e2e330580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe613ad4ba840a9b29cc41530b51a0d', 'request_id': 'req-1930d2ed-bb0c-4852-b931-562e2e330580'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:12.638923 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1930d2ed-bb0c-4852-b931-562e2e330580 admin admin] The token cache is being invalidate because project 0fe613ad4ba840a9b29cc41530b51a0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:12.639438 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1930d2ed-bb0c-4852-b931-562e2e330580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:12.641751 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-1930d2ed-bb0c-4852-b931-562e2e330580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:12.650399 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0253d-0c4a-4146-8e1d-95ac116365e0 service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:12.650946 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1394/2759] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:12 2022] DELETE /identity/v3/projects/0fe613ad4ba840a9b29cc41530b51a0d => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:12.663020 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c0253d-0c4a-4146-8e1d-95ac116365e0 service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:12.778171 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:12.830070 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d5cca-47f9-413f-b6f3-b2fe5867c762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:12.830655 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d5cca-47f9-413f-b6f3-b2fe5867c762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:12.831206 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d5cca-47f9-413f-b6f3-b2fe5867c762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:12.831559 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d5cca-47f9-413f-b6f3-b2fe5867c762 admin admin] PATH_INFO: `/v3/users/6efb7eea519e44e098088f0dd54e1f96` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:12.832987 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d5cca-47f9-413f-b6f3-b2fe5867c762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6efb7eea519e44e098088f0dd54e1f96)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:12.843208 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d5cca-47f9-413f-b6f3-b2fe5867c762 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:12.921528 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d53d5cca-47f9-413f-b6f3-b2fe5867c762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6efb7eea519e44e098088f0dd54e1f96'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:12.922327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d53d5cca-47f9-413f-b6f3-b2fe5867c762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6efb7eea519e44e098088f0dd54e1f96'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:12.922956 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d53d5cca-47f9-413f-b6f3-b2fe5867c762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6efb7eea519e44e098088f0dd54e1f96'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:12.940012 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d53d5cca-47f9-413f-b6f3-b2fe5867c762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6efb7eea519e44e098088f0dd54e1f96'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:12.975959 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d53d5cca-47f9-413f-b6f3-b2fe5867c762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6efb7eea519e44e098088f0dd54e1f96'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:13.038060 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1366/2760] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:12 2022] DELETE /identity/v3/users/6efb7eea519e44e098088f0dd54e1f96 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:13.655444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930d2ed-bb0c-4852-b931-562e2e330580 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:13.656981 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1930d2ed-bb0c-4852-b931-562e2e330580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:13.682295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930d2ed-bb0c-4852-b931-562e2e330580 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:13.697800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:13.762256 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2858c-fe60-4001-a694-2d34906d787f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:13.763936 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2858c-fe60-4001-a694-2d34906d787f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:13.764237 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2858c-fe60-4001-a694-2d34906d787f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:13.764520 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2858c-fe60-4001-a694-2d34906d787f admin admin] PATH_INFO: `/v3/projects/06f034d318f04d05a3112f979dcb754a` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:13.767991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2858c-fe60-4001-a694-2d34906d787f admin admin] RBAC: Authorizing `identity:delete_project(project_id=06f034d318f04d05a3112f979dcb754a)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:13.772107 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2858c-fe60-4001-a694-2d34906d787f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:13.837393 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-eda2858c-fe60-4001-a694-2d34906d787f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06f034d318f04d05a3112f979dcb754a', 'request_id': 'req-eda2858c-fe60-4001-a694-2d34906d787f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:13.845826 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-eda2858c-fe60-4001-a694-2d34906d787f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06f034d318f04d05a3112f979dcb754a', 'request_id': 'req-eda2858c-fe60-4001-a694-2d34906d787f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:13.856086 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-eda2858c-fe60-4001-a694-2d34906d787f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06f034d318f04d05a3112f979dcb754a', 'request_id': 'req-eda2858c-fe60-4001-a694-2d34906d787f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:13.868365 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-eda2858c-fe60-4001-a694-2d34906d787f admin admin] The token cache is being invalidate because project 06f034d318f04d05a3112f979dcb754a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:13.868793 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-eda2858c-fe60-4001-a694-2d34906d787f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:13.870204 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-eda2858c-fe60-4001-a694-2d34906d787f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:13.873325 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1395/2761] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:13 2022] DELETE /identity/v3/projects/06f034d318f04d05a3112f979dcb754a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:13.878059 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f3719-302c-4ce0-86a0-c0a67ac6b333 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:13.878344 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f3719-302c-4ce0-86a0-c0a67ac6b333 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:13.878786 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f3719-302c-4ce0-86a0-c0a67ac6b333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:13.900515 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-ea0f3719-302c-4ce0-86a0-c0a67ac6b333 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:13.901878 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-ea0f3719-302c-4ce0-86a0-c0a67ac6b333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:13.978141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1367/2762] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:13.982729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89e512-503a-4df5-a58d-0872c7c51865 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:13.983029 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89e512-503a-4df5-a58d-0872c7c51865 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:13.984433 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89e512-503a-4df5-a58d-0872c7c51865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:14.045646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6b89e512-503a-4df5-a58d-0872c7c51865 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:14.048955 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6b89e512-503a-4df5-a58d-0872c7c51865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:14.077061 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1396/2763] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:14.078347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f3719-302c-4ce0-86a0-c0a67ac6b333 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:14.094515 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f3719-302c-4ce0-86a0-c0a67ac6b333 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:14.109718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0279f-d105-4f81-8741-d5fd52a297d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:14.111398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0279f-d105-4f81-8741-d5fd52a297d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:14.111858 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0279f-d105-4f81-8741-d5fd52a297d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:14.112368 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0279f-d105-4f81-8741-d5fd52a297d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:14.113635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0279f-d105-4f81-8741-d5fd52a297d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:14.113635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0279f-d105-4f81-8741-d5fd52a297d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:14.118007 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0279f-d105-4f81-8741-d5fd52a297d1 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:14.162262 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1368/2764] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:40:14.163478 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89e512-503a-4df5-a58d-0872c7c51865 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:14.176714 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89e512-503a-4df5-a58d-0872c7c51865 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:14.191528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db0dbc-030c-4a5c-ba3b-0c11a5cda47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:14.192425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db0dbc-030c-4a5c-ba3b-0c11a5cda47c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:14.192801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db0dbc-030c-4a5c-ba3b-0c11a5cda47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:14.193187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db0dbc-030c-4a5c-ba3b-0c11a5cda47c admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:14.194548 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db0dbc-030c-4a5c-ba3b-0c11a5cda47c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:14.196644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db0dbc-030c-4a5c-ba3b-0c11a5cda47c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:14.268915 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1397/2765] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:14 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:14.270635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0279f-d105-4f81-8741-d5fd52a297d1 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:14.285846 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0279f-d105-4f81-8741-d5fd52a297d1 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:14.407714 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:14.427301 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa6aba-d0a6-415d-a3e4-7a4747c1ba34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:14.428224 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa6aba-d0a6-415d-a3e4-7a4747c1ba34 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:14.428641 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa6aba-d0a6-415d-a3e4-7a4747c1ba34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:14.428931 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa6aba-d0a6-415d-a3e4-7a4747c1ba34 admin admin] PATH_INFO: `/v3/users` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:14.429887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa6aba-d0a6-415d-a3e4-7a4747c1ba34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:14.433118 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa6aba-d0a6-415d-a3e4-7a4747c1ba34 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:14.488853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1369/2766] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:14 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:14.489787 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db0dbc-030c-4a5c-ba3b-0c11a5cda47c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:14.506298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db0dbc-030c-4a5c-ba3b-0c11a5cda47c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:14.518662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b301c-410e-43a9-b3ce-44d73bad6521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:14.519295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b301c-410e-43a9-b3ce-44d73bad6521 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:14.519581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b301c-410e-43a9-b3ce-44d73bad6521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:14.519927 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b301c-410e-43a9-b3ce-44d73bad6521 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:14.520592 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b301c-410e-43a9-b3ce-44d73bad6521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:14.522213 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b301c-410e-43a9-b3ce-44d73bad6521 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:14.557187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1398/2767] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:14.560293 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa6aba-d0a6-415d-a3e4-7a4747c1ba34 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:14.575365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa6aba-d0a6-415d-a3e4-7a4747c1ba34 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:14.582384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d522f65-d2e2-48e1-8456-caebb6f73fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:14.583133 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d522f65-d2e2-48e1-8456-caebb6f73fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:14.583416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d522f65-d2e2-48e1-8456-caebb6f73fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:14.583718 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d522f65-d2e2-48e1-8456-caebb6f73fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:14.584440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d522f65-d2e2-48e1-8456-caebb6f73fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:14.586644 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d522f65-d2e2-48e1-8456-caebb6f73fa9 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:14.599946 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1370/2768] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:14.603904 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b301c-410e-43a9-b3ce-44d73bad6521 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:14.615644 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b301c-410e-43a9-b3ce-44d73bad6521 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:14.621857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8164c0-ffec-4f33-af37-6ff46d503c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:14.623118 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8164c0-ffec-4f33-af37-6ff46d503c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:14.623408 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8164c0-ffec-4f33-af37-6ff46d503c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:14.623722 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8164c0-ffec-4f33-af37-6ff46d503c0a admin admin] PATH_INFO: `/v3/projects/7ecc6da53cc74bc78b8deeb628c3bbf9/users/ead8246e315c49fd9940496e6e737fbb/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:14.644158 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8164c0-ffec-4f33-af37-6ff46d503c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ecc6da53cc74bc78b8deeb628c3bbf9, user_id=ead8246e315c49fd9940496e6e737fbb, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:14.646860 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8164c0-ffec-4f33-af37-6ff46d503c0a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:14.662295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1399/2769] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:14 2022] PUT /identity/v3/projects/7ecc6da53cc74bc78b8deeb628c3bbf9/users/ead8246e315c49fd9940496e6e737fbb/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:14.674142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc38fb-ab15-4f43-87fb-27e5ea8386b7 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:14.674142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc38fb-ab15-4f43-87fb-27e5ea8386b7 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:14.674142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc38fb-ab15-4f43-87fb-27e5ea8386b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:14.725429 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-bdbc38fb-ab15-4f43-87fb-27e5ea8386b7 None None] MFA Rules not processed for user `ead8246e315c49fd9940496e6e737fbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:14.727540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-bdbc38fb-ab15-4f43-87fb-27e5ea8386b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:14.824496 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1371/2770] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:14.838716 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8164c0-ffec-4f33-af37-6ff46d503c0a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:14.841895 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2f8164c0-ffec-4f33-af37-6ff46d503c0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:14.942602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:14.973187 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8164c0-ffec-4f33-af37-6ff46d503c0a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:14.983657 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf1d33-06d9-4379-acbd-8327f7411905 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:14.984354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf1d33-06d9-4379-acbd-8327f7411905 service cinder] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:14.984780 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf1d33-06d9-4379-acbd-8327f7411905 service cinder] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:14.985258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf1d33-06d9-4379-acbd-8327f7411905 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:14.998311 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf1d33-06d9-4379-acbd-8327f7411905 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:14.999854 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaf1d33-06d9-4379-acbd-8327f7411905 service cinder] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:15.011211 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1400/2771] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:15.677193 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc38fb-ab15-4f43-87fb-27e5ea8386b7 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:15.688129 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc38fb-ab15-4f43-87fb-27e5ea8386b7 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:15.692694 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d0078-74bb-4726-acfd-a73f642c40b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:15.693633 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d0078-74bb-4726-acfd-a73f642c40b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:15.694127 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d0078-74bb-4726-acfd-a73f642c40b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:15.694636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d0078-74bb-4726-acfd-a73f642c40b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:15.696861 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-f38d0078-74bb-4726-acfd-a73f642c40b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:15.784589 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d0078-74bb-4726-acfd-a73f642c40b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:15.786176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d0078-74bb-4726-acfd-a73f642c40b7 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:15.799040 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1372/2772] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:24.957224 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf1d33-06d9-4379-acbd-8327f7411905 service cinder] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:24.958674 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-abaf1d33-06d9-4379-acbd-8327f7411905 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:25.028720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaf1d33-06d9-4379-acbd-8327f7411905 service cinder] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:25.033527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcd6e3-c3a1-4c9c-b197-7f1ab2d5e204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:25.034153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcd6e3-c3a1-4c9c-b197-7f1ab2d5e204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:25.034431 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcd6e3-c3a1-4c9c-b197-7f1ab2d5e204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:25.034703 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcd6e3-c3a1-4c9c-b197-7f1ab2d5e204 admin admin] PATH_INFO: `/v3/users/c9c6d6ceb012410ca6e6b1353fc3f22f` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:25.035856 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fcd6e3-c3a1-4c9c-b197-7f1ab2d5e204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9c6d6ceb012410ca6e6b1353fc3f22f)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:25.037771 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fcd6e3-c3a1-4c9c-b197-7f1ab2d5e204 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:25.084111 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-36fcd6e3-c3a1-4c9c-b197-7f1ab2d5e204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9c6d6ceb012410ca6e6b1353fc3f22f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:25.084779 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-36fcd6e3-c3a1-4c9c-b197-7f1ab2d5e204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9c6d6ceb012410ca6e6b1353fc3f22f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:25.091903 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-36fcd6e3-c3a1-4c9c-b197-7f1ab2d5e204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9c6d6ceb012410ca6e6b1353fc3f22f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:25.099421 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-36fcd6e3-c3a1-4c9c-b197-7f1ab2d5e204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9c6d6ceb012410ca6e6b1353fc3f22f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:25.111554 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-36fcd6e3-c3a1-4c9c-b197-7f1ab2d5e204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9c6d6ceb012410ca6e6b1353fc3f22f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:25.117417 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1401/2773] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:24 2022] DELETE /identity/v3/users/c9c6d6ceb012410ca6e6b1353fc3f22f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:25.122002 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26017f14-c375-4e6e-a0ee-42dbf8cef9b2 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:25.123083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26017f14-c375-4e6e-a0ee-42dbf8cef9b2 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:25.123083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26017f14-c375-4e6e-a0ee-42dbf8cef9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:25.142132 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-26017f14-c375-4e6e-a0ee-42dbf8cef9b2 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:25.143391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-26017f14-c375-4e6e-a0ee-42dbf8cef9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:25.194740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:25.209139 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1373/2774] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:25.218963 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcd6e3-c3a1-4c9c-b197-7f1ab2d5e204 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:25.220416 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-36fcd6e3-c3a1-4c9c-b197-7f1ab2d5e204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:25.281567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcd6e3-c3a1-4c9c-b197-7f1ab2d5e204 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:25.285815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a7ed7-0f4d-443a-bec5-6793ace79044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:25.286404 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a7ed7-0f4d-443a-bec5-6793ace79044 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:25.286686 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a7ed7-0f4d-443a-bec5-6793ace79044 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:25.286961 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a7ed7-0f4d-443a-bec5-6793ace79044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:25.298881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823a7ed7-0f4d-443a-bec5-6793ace79044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:25.300200 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823a7ed7-0f4d-443a-bec5-6793ace79044 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:25.315779 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1402/2775] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:25.603580 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26017f14-c375-4e6e-a0ee-42dbf8cef9b2 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:25.605176 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-26017f14-c375-4e6e-a0ee-42dbf8cef9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:25.618491 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26017f14-c375-4e6e-a0ee-42dbf8cef9b2 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:25.623746 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:25.624385 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:25.624669 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:25.624944 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5 admin admin] PATH_INFO: `/v3/projects/1a214a1227ae426ea4270b54904c7880` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:25.626189 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a214a1227ae426ea4270b54904c7880)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:25.628146 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:25.675682 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a214a1227ae426ea4270b54904c7880', 'request_id': 'req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:25.681148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a214a1227ae426ea4270b54904c7880', 'request_id': 'req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:25.690453 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a214a1227ae426ea4270b54904c7880', 'request_id': 'req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:25.702972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5 admin admin] The token cache is being invalidate because project 1a214a1227ae426ea4270b54904c7880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:25.703268 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:25.703941 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:25.705427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1374/2776] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:25 2022] DELETE /identity/v3/projects/1a214a1227ae426ea4270b54904c7880 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:25.709377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a7ed7-0f4d-443a-bec5-6793ace79044 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:25.714214 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:25.721762 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a7ed7-0f4d-443a-bec5-6793ace79044 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:25.790440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f7fb8-9bc3-420f-9879-2632a8d266b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:25.791049 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f7fb8-9bc3-420f-9879-2632a8d266b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:25.791335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f7fb8-9bc3-420f-9879-2632a8d266b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:25.791617 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f7fb8-9bc3-420f-9879-2632a8d266b8 admin admin] PATH_INFO: `/v3/users/72fceb59be78425d8fceefecfce1e254` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:25.793001 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f7fb8-9bc3-420f-9879-2632a8d266b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72fceb59be78425d8fceefecfce1e254)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:25.796221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f7fb8-9bc3-420f-9879-2632a8d266b8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:25.869876 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c45f7fb8-9bc3-420f-9879-2632a8d266b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72fceb59be78425d8fceefecfce1e254'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:25.870667 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c45f7fb8-9bc3-420f-9879-2632a8d266b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72fceb59be78425d8fceefecfce1e254'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:25.887019 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c45f7fb8-9bc3-420f-9879-2632a8d266b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72fceb59be78425d8fceefecfce1e254'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:25.898906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c45f7fb8-9bc3-420f-9879-2632a8d266b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72fceb59be78425d8fceefecfce1e254'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:25.913338 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-c45f7fb8-9bc3-420f-9879-2632a8d266b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72fceb59be78425d8fceefecfce1e254'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:25.920028 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1403/2777] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:25 2022] DELETE /identity/v3/users/72fceb59be78425d8fceefecfce1e254 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:26.325251 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:26.328923 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:26.347754 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b0d1a-a8d0-4bce-961d-2bda1d7c9ca5 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:26.418080 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d56a9f-791f-40d3-a54c-69ec8e1e9fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:26.419050 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d56a9f-791f-40d3-a54c-69ec8e1e9fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:26.419698 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d56a9f-791f-40d3-a54c-69ec8e1e9fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:26.420160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d56a9f-791f-40d3-a54c-69ec8e1e9fe3 admin admin] PATH_INFO: `/v3/projects/6d33e517a4cc4e2c9bbb3387cd81f19f` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:26.422148 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d56a9f-791f-40d3-a54c-69ec8e1e9fe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d33e517a4cc4e2c9bbb3387cd81f19f)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:26.434565 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d56a9f-791f-40d3-a54c-69ec8e1e9fe3 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:26.482342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-52d56a9f-791f-40d3-a54c-69ec8e1e9fe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d33e517a4cc4e2c9bbb3387cd81f19f', 'request_id': 'req-52d56a9f-791f-40d3-a54c-69ec8e1e9fe3'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:26.488119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-52d56a9f-791f-40d3-a54c-69ec8e1e9fe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d33e517a4cc4e2c9bbb3387cd81f19f', 'request_id': 'req-52d56a9f-791f-40d3-a54c-69ec8e1e9fe3'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:26.496867 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-52d56a9f-791f-40d3-a54c-69ec8e1e9fe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d33e517a4cc4e2c9bbb3387cd81f19f', 'request_id': 'req-52d56a9f-791f-40d3-a54c-69ec8e1e9fe3'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:26.512155 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-52d56a9f-791f-40d3-a54c-69ec8e1e9fe3 admin admin] The token cache is being invalidate because project 6d33e517a4cc4e2c9bbb3387cd81f19f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:26.512687 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-52d56a9f-791f-40d3-a54c-69ec8e1e9fe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:26.514002 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-52d56a9f-791f-40d3-a54c-69ec8e1e9fe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:26.515655 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1375/2778] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:26 2022] DELETE /identity/v3/projects/6d33e517a4cc4e2c9bbb3387cd81f19f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:26.521845 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87286dc6-e968-4643-a7b0-5ce615ec1461 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:26.522319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87286dc6-e968-4643-a7b0-5ce615ec1461 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:26.522567 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87286dc6-e968-4643-a7b0-5ce615ec1461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:26.543914 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-87286dc6-e968-4643-a7b0-5ce615ec1461 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:26.545267 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-87286dc6-e968-4643-a7b0-5ce615ec1461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:26.618687 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1404/2779] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:26.628355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717a3aa-1f77-4688-8fe3-830a5fc9c788 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:26.628627 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717a3aa-1f77-4688-8fe3-830a5fc9c788 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:26.629137 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717a3aa-1f77-4688-8fe3-830a5fc9c788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:26.646586 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:26.653519 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-d717a3aa-1f77-4688-8fe3-830a5fc9c788 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:26.654968 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d717a3aa-1f77-4688-8fe3-830a5fc9c788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:26.666962 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1376/2780] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:26.672326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87286dc6-e968-4643-a7b0-5ce615ec1461 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:26.683458 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87286dc6-e968-4643-a7b0-5ce615ec1461 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:26.689492 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00dfaef-b6a9-493b-8a79-4d82faf97afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:26.690131 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dfaef-b6a9-493b-8a79-4d82faf97afe admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:26.690413 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dfaef-b6a9-493b-8a79-4d82faf97afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:26.690691 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dfaef-b6a9-493b-8a79-4d82faf97afe admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:26.691357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00dfaef-b6a9-493b-8a79-4d82faf97afe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:26.691847 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00dfaef-b6a9-493b-8a79-4d82faf97afe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:26.694219 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00dfaef-b6a9-493b-8a79-4d82faf97afe admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:26.711832 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1405/2781] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:40:26.713353 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717a3aa-1f77-4688-8fe3-830a5fc9c788 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:26.726780 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717a3aa-1f77-4688-8fe3-830a5fc9c788 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:26.733351 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d883660-e97f-478c-b2cf-9c5d105d28cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:26.734972 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d883660-e97f-478c-b2cf-9c5d105d28cb admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:26.735268 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d883660-e97f-478c-b2cf-9c5d105d28cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:26.735598 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d883660-e97f-478c-b2cf-9c5d105d28cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:26.737434 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d883660-e97f-478c-b2cf-9c5d105d28cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:26.739772 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d883660-e97f-478c-b2cf-9c5d105d28cb admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:26.776755 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1377/2782] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:26 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:26.781009 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00dfaef-b6a9-493b-8a79-4d82faf97afe admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:26.792571 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00dfaef-b6a9-493b-8a79-4d82faf97afe admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:26.798981 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:26.877743 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04040504-eefd-4440-b1a2-80ac4f3e8e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:26.878418 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04040504-eefd-4440-b1a2-80ac4f3e8e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:26.878709 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04040504-eefd-4440-b1a2-80ac4f3e8e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:26.878989 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04040504-eefd-4440-b1a2-80ac4f3e8e5d admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:26.879641 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04040504-eefd-4440-b1a2-80ac4f3e8e5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:26.881913 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04040504-eefd-4440-b1a2-80ac4f3e8e5d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:26.906675 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1406/2783] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:26 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:26.910801 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d883660-e97f-478c-b2cf-9c5d105d28cb admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:26.922081 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d883660-e97f-478c-b2cf-9c5d105d28cb admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:26.928137 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698e12e-f93c-488e-a7e0-04e3bc59b5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:26.928779 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698e12e-f93c-488e-a7e0-04e3bc59b5bd admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:26.929067 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698e12e-f93c-488e-a7e0-04e3bc59b5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:26.929387 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698e12e-f93c-488e-a7e0-04e3bc59b5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:26.930074 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698e12e-f93c-488e-a7e0-04e3bc59b5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:26.931708 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7698e12e-f93c-488e-a7e0-04e3bc59b5bd admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:26.946260 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1378/2784] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:40:26.947436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04040504-eefd-4440-b1a2-80ac4f3e8e5d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:26.961794 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04040504-eefd-4440-b1a2-80ac4f3e8e5d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:26.968088 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8e415-7292-45f5-84c8-68608c1440f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:26.968710 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8e415-7292-45f5-84c8-68608c1440f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:26.968993 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8e415-7292-45f5-84c8-68608c1440f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:26.969271 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8e415-7292-45f5-84c8-68608c1440f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:26.969938 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8e415-7292-45f5-84c8-68608c1440f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:26.971669 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8e415-7292-45f5-84c8-68608c1440f2 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:26.984772 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1407/2785] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:26.989141 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698e12e-f93c-488e-a7e0-04e3bc59b5bd admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:26.999571 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698e12e-f93c-488e-a7e0-04e3bc59b5bd admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:27.004251 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab5564-ebad-496a-93a3-07fb0ac0a859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:27.004913 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab5564-ebad-496a-93a3-07fb0ac0a859 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:27.005198 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab5564-ebad-496a-93a3-07fb0ac0a859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:27.005476 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab5564-ebad-496a-93a3-07fb0ac0a859 admin admin] PATH_INFO: `/v3/projects/aae123869ee045f3803fdecce86c982e/users/625ea3120ade427c99aa0038d5d20b49/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:27.021892 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab5564-ebad-496a-93a3-07fb0ac0a859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae123869ee045f3803fdecce86c982e, user_id=625ea3120ade427c99aa0038d5d20b49, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:27.024740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ab5564-ebad-496a-93a3-07fb0ac0a859 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:27.040132 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1379/2786] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:26 2022] PUT /identity/v3/projects/aae123869ee045f3803fdecce86c982e/users/625ea3120ade427c99aa0038d5d20b49/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:27.052666 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b196580-014e-4ae2-92ce-ab66fe484307 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:27.053127 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b196580-014e-4ae2-92ce-ab66fe484307 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:27.053570 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b196580-014e-4ae2-92ce-ab66fe484307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:27.084734 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6b196580-014e-4ae2-92ce-ab66fe484307 None None] MFA Rules not processed for user `625ea3120ade427c99aa0038d5d20b49`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:27.085897 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6b196580-014e-4ae2-92ce-ab66fe484307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:27.153052 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1408/2787] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:27.164866 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab5564-ebad-496a-93a3-07fb0ac0a859 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:27.166588 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-19ab5564-ebad-496a-93a3-07fb0ac0a859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:27.223398 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab5564-ebad-496a-93a3-07fb0ac0a859 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:27.227083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79fc1a-6056-4393-ac68-538189b2253f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:27.227722 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79fc1a-6056-4393-ac68-538189b2253f service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:27.228007 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79fc1a-6056-4393-ac68-538189b2253f service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:27.228281 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79fc1a-6056-4393-ac68-538189b2253f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:27.236476 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79fc1a-6056-4393-ac68-538189b2253f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:27.237900 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79fc1a-6056-4393-ac68-538189b2253f service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:27.246577 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1380/2788] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:27.272124 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b196580-014e-4ae2-92ce-ab66fe484307 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:27.342729 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b196580-014e-4ae2-92ce-ab66fe484307 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:27.347580 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f045839-ecb3-444e-a933-5d6f52e0949b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:27.348246 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f045839-ecb3-444e-a933-5d6f52e0949b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:27.348530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f045839-ecb3-444e-a933-5d6f52e0949b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:27.348801 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f045839-ecb3-444e-a933-5d6f52e0949b admin admin] PATH_INFO: `/v3/users/625ea3120ade427c99aa0038d5d20b49` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:27.349941 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f045839-ecb3-444e-a933-5d6f52e0949b admin admin] RBAC: Authorizing `identity:delete_user(user_id=625ea3120ade427c99aa0038d5d20b49)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:27.351901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f045839-ecb3-444e-a933-5d6f52e0949b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:27.374518 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:27.401456 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2f045839-ecb3-444e-a933-5d6f52e0949b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '625ea3120ade427c99aa0038d5d20b49'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:27.402114 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2f045839-ecb3-444e-a933-5d6f52e0949b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '625ea3120ade427c99aa0038d5d20b49'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:27.409208 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2f045839-ecb3-444e-a933-5d6f52e0949b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '625ea3120ade427c99aa0038d5d20b49'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:27.416515 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2f045839-ecb3-444e-a933-5d6f52e0949b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '625ea3120ade427c99aa0038d5d20b49'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:27.428731 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2f045839-ecb3-444e-a933-5d6f52e0949b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '625ea3120ade427c99aa0038d5d20b49'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:27.434550 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1409/2789] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:27 2022] DELETE /identity/v3/users/625ea3120ade427c99aa0038d5d20b49 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:27.439724 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccea93d-bfaf-41ba-b1c1-8f1cd6f6d754 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:27.440129 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccea93d-bfaf-41ba-b1c1-8f1cd6f6d754 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:27.440277 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccea93d-bfaf-41ba-b1c1-8f1cd6f6d754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:27.460090 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-eccea93d-bfaf-41ba-b1c1-8f1cd6f6d754 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:27.461300 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-eccea93d-bfaf-41ba-b1c1-8f1cd6f6d754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:27.521088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1381/2790] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:27.530797 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f045839-ecb3-444e-a933-5d6f52e0949b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:27.532245 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2f045839-ecb3-444e-a933-5d6f52e0949b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:27.590978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f045839-ecb3-444e-a933-5d6f52e0949b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:27.594834 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d0da6-b900-428c-812c-d53384e7c934 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:27.595379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d0da6-b900-428c-812c-d53384e7c934 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:27.595800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d0da6-b900-428c-812c-d53384e7c934 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:27.596075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d0da6-b900-428c-812c-d53384e7c934 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:27.604974 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d0da6-b900-428c-812c-d53384e7c934 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:27.606436 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d0da6-b900-428c-812c-d53384e7c934 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:27.616469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1410/2791] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:27.883870 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccea93d-bfaf-41ba-b1c1-8f1cd6f6d754 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:27.885369 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-eccea93d-bfaf-41ba-b1c1-8f1cd6f6d754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:27.904564 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccea93d-bfaf-41ba-b1c1-8f1cd6f6d754 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:27.909076 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5ed95-18f0-4c23-9499-c3351c920e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:27.909704 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5ed95-18f0-4c23-9499-c3351c920e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:27.909978 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5ed95-18f0-4c23-9499-c3351c920e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:27.910744 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5ed95-18f0-4c23-9499-c3351c920e2b admin admin] PATH_INFO: `/v3/projects/aae123869ee045f3803fdecce86c982e` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:27.912125 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5ed95-18f0-4c23-9499-c3351c920e2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aae123869ee045f3803fdecce86c982e)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:27.914032 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5ed95-18f0-4c23-9499-c3351c920e2b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:27.957951 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:27.965420 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6cd5ed95-18f0-4c23-9499-c3351c920e2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aae123869ee045f3803fdecce86c982e', 'request_id': 'req-6cd5ed95-18f0-4c23-9499-c3351c920e2b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:27.972401 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6cd5ed95-18f0-4c23-9499-c3351c920e2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aae123869ee045f3803fdecce86c982e', 'request_id': 'req-6cd5ed95-18f0-4c23-9499-c3351c920e2b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:27.979769 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6cd5ed95-18f0-4c23-9499-c3351c920e2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aae123869ee045f3803fdecce86c982e', 'request_id': 'req-6cd5ed95-18f0-4c23-9499-c3351c920e2b'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:27.992319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6cd5ed95-18f0-4c23-9499-c3351c920e2b admin admin] The token cache is being invalidate because project aae123869ee045f3803fdecce86c982e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:27.992636 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6cd5ed95-18f0-4c23-9499-c3351c920e2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:27.993270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-6cd5ed95-18f0-4c23-9499-c3351c920e2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:27.994749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1382/2792] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:27 2022] DELETE /identity/v3/projects/aae123869ee045f3803fdecce86c982e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:27.999831 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba6e5b-462d-44e6-bd76-cd25a603b837 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:28.000144 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba6e5b-462d-44e6-bd76-cd25a603b837 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:28.000437 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba6e5b-462d-44e6-bd76-cd25a603b837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:28.019561 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-e8ba6e5b-462d-44e6-bd76-cd25a603b837 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:28.020777 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-e8ba6e5b-462d-44e6-bd76-cd25a603b837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:28.080900 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1411/2793] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:28.088554 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee43488-2e38-4265-8eb9-9f9e93798fb5 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:28.088840 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee43488-2e38-4265-8eb9-9f9e93798fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:28.089245 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee43488-2e38-4265-8eb9-9f9e93798fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:28.109817 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-bee43488-2e38-4265-8eb9-9f9e93798fb5 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:28.111004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-bee43488-2e38-4265-8eb9-9f9e93798fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:28.119172 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1383/2794] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:28.123653 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba6e5b-462d-44e6-bd76-cd25a603b837 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:28.133629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba6e5b-462d-44e6-bd76-cd25a603b837 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:28.138444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23053d3-43d5-477a-9b1a-cdb4a8587319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:28.139047 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23053d3-43d5-477a-9b1a-cdb4a8587319 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:28.139331 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23053d3-43d5-477a-9b1a-cdb4a8587319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:28.139610 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23053d3-43d5-477a-9b1a-cdb4a8587319 admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:28.140291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23053d3-43d5-477a-9b1a-cdb4a8587319 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:28.140597 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23053d3-43d5-477a-9b1a-cdb4a8587319 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:28.142247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23053d3-43d5-477a-9b1a-cdb4a8587319 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:28.155837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1412/2795] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:40:28.160748 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee43488-2e38-4265-8eb9-9f9e93798fb5 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:28.170750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee43488-2e38-4265-8eb9-9f9e93798fb5 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:28.175761 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4beebae-1c21-4bda-9417-bb6f69c32640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:28.176368 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4beebae-1c21-4bda-9417-bb6f69c32640 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:28.176783 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4beebae-1c21-4bda-9417-bb6f69c32640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:28.177062 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4beebae-1c21-4bda-9417-bb6f69c32640 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:28.177696 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4beebae-1c21-4bda-9417-bb6f69c32640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:28.179599 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4beebae-1c21-4bda-9417-bb6f69c32640 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:28.210083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1384/2796] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:28 2022] POST /identity/v3/projects => generated 397 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:28.214288 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23053d3-43d5-477a-9b1a-cdb4a8587319 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:28.225401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23053d3-43d5-477a-9b1a-cdb4a8587319 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:28.250310 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:28.301697 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f40f02-8579-4f0c-98e5-64ed9ee34b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:28.302467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f40f02-8579-4f0c-98e5-64ed9ee34b70 admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:28.303135 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f40f02-8579-4f0c-98e5-64ed9ee34b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:28.303584 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f40f02-8579-4f0c-98e5-64ed9ee34b70 admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:28.304966 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f40f02-8579-4f0c-98e5-64ed9ee34b70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:28.307439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f40f02-8579-4f0c-98e5-64ed9ee34b70 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:28.334000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1413/2797] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:28 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:28.335285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4beebae-1c21-4bda-9417-bb6f69c32640 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:28.346815 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4beebae-1c21-4bda-9417-bb6f69c32640 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:28.351439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74408710-b278-473f-81bd-cdded770dc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:28.352166 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74408710-b278-473f-81bd-cdded770dc19 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:28.352442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74408710-b278-473f-81bd-cdded770dc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:28.352714 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74408710-b278-473f-81bd-cdded770dc19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:28.353357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74408710-b278-473f-81bd-cdded770dc19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:28.354949 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74408710-b278-473f-81bd-cdded770dc19 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:28.364982 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1385/2798] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:28.368872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f40f02-8579-4f0c-98e5-64ed9ee34b70 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:28.379228 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f40f02-8579-4f0c-98e5-64ed9ee34b70 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:28.383934 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a08c72-8696-4a4b-8d10-e2bdd3cc279a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:28.384525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a08c72-8696-4a4b-8d10-e2bdd3cc279a admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:28.384807 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a08c72-8696-4a4b-8d10-e2bdd3cc279a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:28.385080 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a08c72-8696-4a4b-8d10-e2bdd3cc279a admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:28.385720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a08c72-8696-4a4b-8d10-e2bdd3cc279a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:28.387305 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a08c72-8696-4a4b-8d10-e2bdd3cc279a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:28.398488 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1414/2799] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:28.402333 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74408710-b278-473f-81bd-cdded770dc19 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:28.412869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74408710-b278-473f-81bd-cdded770dc19 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:28.417455 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fe670-35a9-412e-a129-1a50b243147b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:28.418088 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fe670-35a9-412e-a129-1a50b243147b admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:28.418370 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fe670-35a9-412e-a129-1a50b243147b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:28.418645 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fe670-35a9-412e-a129-1a50b243147b admin admin] PATH_INFO: `/v3/projects/acbb6565224546d48d49a8e3ac621718/users/97adfe5f3f654c52a2a6c180c71b88cc/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:28.432953 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fe670-35a9-412e-a129-1a50b243147b admin admin] RBAC: Authorizing `identity:create_grant(project_id=acbb6565224546d48d49a8e3ac621718, user_id=97adfe5f3f654c52a2a6c180c71b88cc, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:28.435726 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fe670-35a9-412e-a129-1a50b243147b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:28.460403 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1386/2800] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:28 2022] PUT /identity/v3/projects/acbb6565224546d48d49a8e3ac621718/users/97adfe5f3f654c52a2a6c180c71b88cc/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:28.471363 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d92ea-f0fa-40ec-87b5-b3afc5af9f1c None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:28.471688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d92ea-f0fa-40ec-87b5-b3afc5af9f1c None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:28.471972 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d92ea-f0fa-40ec-87b5-b3afc5af9f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:28.510469 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-9a7d92ea-f0fa-40ec-87b5-b3afc5af9f1c None None] MFA Rules not processed for user `97adfe5f3f654c52a2a6c180c71b88cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:28.511715 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9a7d92ea-f0fa-40ec-87b5-b3afc5af9f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:28.595401 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1415/2801] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:28 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:28.606578 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fe670-35a9-412e-a129-1a50b243147b admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:28.608991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d74fe670-35a9-412e-a129-1a50b243147b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:28.667411 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fe670-35a9-412e-a129-1a50b243147b admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:28.671934 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7576d11-028e-4e27-9255-25ac5f92a023 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a42d8e462ba542a5bf58ba052dceeb8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:28.672567 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7576d11-028e-4e27-9255-25ac5f92a023 service cinder] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:28.672850 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7576d11-028e-4e27-9255-25ac5f92a023 service cinder] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:28.673151 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7576d11-028e-4e27-9255-25ac5f92a023 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:28.681451 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7576d11-028e-4e27-9255-25ac5f92a023 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:28.682887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7576d11-028e-4e27-9255-25ac5f92a023 service cinder] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:28.691792 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1387/2802] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:28 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:34.955496 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d92ea-f0fa-40ec-87b5-b3afc5af9f1c None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:34.956950 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-9a7d92ea-f0fa-40ec-87b5-b3afc5af9f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:35.055216 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:35.114220 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d92ea-f0fa-40ec-87b5-b3afc5af9f1c None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:35.119319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be3b72-f55a-4963-8ce9-a3c6f880253b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ba45e3cff74962965b248f9f017f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:35.119952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be3b72-f55a-4963-8ce9-a3c6f880253b service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:35.120236 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be3b72-f55a-4963-8ce9-a3c6f880253b service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:35.120511 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be3b72-f55a-4963-8ce9-a3c6f880253b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:35.122883 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-89be3b72-f55a-4963-8ce9-a3c6f880253b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:35.134912 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be3b72-f55a-4963-8ce9-a3c6f880253b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:35.136534 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be3b72-f55a-4963-8ce9-a3c6f880253b service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:35.147602 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1416/2803] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:36.350671 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db46cb-6c4f-4437-b206-4721557dd20e None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:36.351136 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db46cb-6c4f-4437-b206-4721557dd20e None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:36.351243 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db46cb-6c4f-4437-b206-4721557dd20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:36.370549 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-e1db46cb-6c4f-4437-b206-4721557dd20e None None] MFA Rules not processed for user `25fad82cbf944d7db13d6bf33d3297a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:36.371787 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e1db46cb-6c4f-4437-b206-4721557dd20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:36.421090 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:36.431004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1388/2804] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:42.011397 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be3b72-f55a-4963-8ce9-a3c6f880253b service placement] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:42.103326 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be3b72-f55a-4963-8ce9-a3c6f880253b service placement] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:42.108643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e845b-4ace-49fb-bc57-3811a346bc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:42.109277 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e845b-4ace-49fb-bc57-3811a346bc49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.109559 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e845b-4ace-49fb-bc57-3811a346bc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.109837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e845b-4ace-49fb-bc57-3811a346bc49 admin admin] PATH_INFO: `/v3/users/97adfe5f3f654c52a2a6c180c71b88cc` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.111010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e845b-4ace-49fb-bc57-3811a346bc49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97adfe5f3f654c52a2a6c180c71b88cc)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:42.112982 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e845b-4ace-49fb-bc57-3811a346bc49 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:42.166430 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fa5e845b-4ace-49fb-bc57-3811a346bc49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97adfe5f3f654c52a2a6c180c71b88cc'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.167115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fa5e845b-4ace-49fb-bc57-3811a346bc49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97adfe5f3f654c52a2a6c180c71b88cc'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.177704 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fa5e845b-4ace-49fb-bc57-3811a346bc49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97adfe5f3f654c52a2a6c180c71b88cc'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.188198 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fa5e845b-4ace-49fb-bc57-3811a346bc49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97adfe5f3f654c52a2a6c180c71b88cc'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.203377 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-fa5e845b-4ace-49fb-bc57-3811a346bc49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97adfe5f3f654c52a2a6c180c71b88cc'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.214600 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1417/2805] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:42 2022] DELETE /identity/v3/users/97adfe5f3f654c52a2a6c180c71b88cc => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:42.219699 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84457237-cfa3-4345-8796-3ca8616410a5 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.220199 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84457237-cfa3-4345-8796-3ca8616410a5 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.220315 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84457237-cfa3-4345-8796-3ca8616410a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.259217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-84457237-cfa3-4345-8796-3ca8616410a5 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:42.260528 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-84457237-cfa3-4345-8796-3ca8616410a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:42.281123 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e845b-4ace-49fb-bc57-3811a346bc49 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:42.282747 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-fa5e845b-4ace-49fb-bc57-3811a346bc49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:42.312637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:42.343319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1389/2806] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:42.353066 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84457237-cfa3-4345-8796-3ca8616410a5 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:42.355521 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-84457237-cfa3-4345-8796-3ca8616410a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:42.379350 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e845b-4ace-49fb-bc57-3811a346bc49 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:42.385215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43798009-b4ed-4eaa-a537-bef459053a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:42.385846 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43798009-b4ed-4eaa-a537-bef459053a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.386161 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43798009-b4ed-4eaa-a537-bef459053a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.386419 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43798009-b4ed-4eaa-a537-bef459053a1b admin admin] PATH_INFO: `/v3/users/ead8246e315c49fd9940496e6e737fbb` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.387586 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43798009-b4ed-4eaa-a537-bef459053a1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ead8246e315c49fd9940496e6e737fbb)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:42.391556 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43798009-b4ed-4eaa-a537-bef459053a1b admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:42.428899 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84457237-cfa3-4345-8796-3ca8616410a5 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:42.436298 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e021b5-32a1-4d38-bda4-b4210d1fffda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:42.436920 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e021b5-32a1-4d38-bda4-b4210d1fffda service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.437200 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e021b5-32a1-4d38-bda4-b4210d1fffda service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.437471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e021b5-32a1-4d38-bda4-b4210d1fffda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.452404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e021b5-32a1-4d38-bda4-b4210d1fffda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:42.455228 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e021b5-32a1-4d38-bda4-b4210d1fffda service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:42.456008 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-43798009-b4ed-4eaa-a537-bef459053a1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ead8246e315c49fd9940496e6e737fbb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.456863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-43798009-b4ed-4eaa-a537-bef459053a1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ead8246e315c49fd9940496e6e737fbb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.468285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-43798009-b4ed-4eaa-a537-bef459053a1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ead8246e315c49fd9940496e6e737fbb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.470772 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1390/2807] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:40:42.485829 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-43798009-b4ed-4eaa-a537-bef459053a1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ead8246e315c49fd9940496e6e737fbb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.512175 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-43798009-b4ed-4eaa-a537-bef459053a1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ead8246e315c49fd9940496e6e737fbb'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:42.527813 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1418/2808] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:40:42 2022] DELETE /identity/v3/users/ead8246e315c49fd9940496e6e737fbb => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:42.532434 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515abb9-a68d-418e-82a6-17964d8331c9 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.532834 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515abb9-a68d-418e-82a6-17964d8331c9 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.533005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515abb9-a68d-418e-82a6-17964d8331c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.573461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-1515abb9-a68d-418e-82a6-17964d8331c9 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:42.576295 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1515abb9-a68d-418e-82a6-17964d8331c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:42.673956 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1391/2809] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:42.694702 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43798009-b4ed-4eaa-a537-bef459053a1b admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:42.696379 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-43798009-b4ed-4eaa-a537-bef459053a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:42.786410 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43798009-b4ed-4eaa-a537-bef459053a1b admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:42.790670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb281a-028e-49c3-b793-5a1911c98b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:42.791425 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb281a-028e-49c3-b793-5a1911c98b0f service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:42.791901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb281a-028e-49c3-b793-5a1911c98b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:42.792314 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb281a-028e-49c3-b793-5a1911c98b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:42.805678 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb281a-028e-49c3-b793-5a1911c98b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:42.808304 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb281a-028e-49c3-b793-5a1911c98b0f service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:42.824471 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1419/2810] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:43.100891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515abb9-a68d-418e-82a6-17964d8331c9 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:43.103337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1515abb9-a68d-418e-82a6-17964d8331c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:43.117050 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515abb9-a68d-418e-82a6-17964d8331c9 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:43.123460 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6e97d-5fae-4743-b342-94ebd191e500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:43.124436 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6e97d-5fae-4743-b342-94ebd191e500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.136158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6e97d-5fae-4743-b342-94ebd191e500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.137875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6e97d-5fae-4743-b342-94ebd191e500 admin admin] PATH_INFO: `/v3/projects/acbb6565224546d48d49a8e3ac621718` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.137875 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6e97d-5fae-4743-b342-94ebd191e500 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acbb6565224546d48d49a8e3ac621718)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:43.141046 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6e97d-5fae-4743-b342-94ebd191e500 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:43.203163 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:43.220570 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-29a6e97d-5fae-4743-b342-94ebd191e500 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acbb6565224546d48d49a8e3ac621718', 'request_id': 'req-29a6e97d-5fae-4743-b342-94ebd191e500'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:43.227217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-29a6e97d-5fae-4743-b342-94ebd191e500 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acbb6565224546d48d49a8e3ac621718', 'request_id': 'req-29a6e97d-5fae-4743-b342-94ebd191e500'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:43.238444 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-29a6e97d-5fae-4743-b342-94ebd191e500 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acbb6565224546d48d49a8e3ac621718', 'request_id': 'req-29a6e97d-5fae-4743-b342-94ebd191e500'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:43.259278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-29a6e97d-5fae-4743-b342-94ebd191e500 admin admin] The token cache is being invalidate because project acbb6565224546d48d49a8e3ac621718 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:43.260044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-29a6e97d-5fae-4743-b342-94ebd191e500 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:43.260709 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-29a6e97d-5fae-4743-b342-94ebd191e500 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:43.262241 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1392/2811] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:43 2022] DELETE /identity/v3/projects/acbb6565224546d48d49a8e3ac621718 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:43.268629 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42907e5e-4aa3-4586-9f50-4cb85bddff5d None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.269158 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42907e5e-4aa3-4586-9f50-4cb85bddff5d None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.269259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42907e5e-4aa3-4586-9f50-4cb85bddff5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.328766 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-42907e5e-4aa3-4586-9f50-4cb85bddff5d None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:43.330082 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-42907e5e-4aa3-4586-9f50-4cb85bddff5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:43.341291 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:43.411260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1420/2812] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:43.413147 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b4290-e5af-4461-b44b-30720a15994f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.413640 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b4290-e5af-4461-b44b-30720a15994f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.414104 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b4290-e5af-4461-b44b-30720a15994f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.441509 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-a87b4290-e5af-4461-b44b-30720a15994f None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:43.443541 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a87b4290-e5af-4461-b44b-30720a15994f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:43.452493 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1393/2813] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:43.456978 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42907e5e-4aa3-4586-9f50-4cb85bddff5d None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:43.471906 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42907e5e-4aa3-4586-9f50-4cb85bddff5d None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:43.477231 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0a70b-492b-4f1b-be45-6f9b2676c32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:43.478346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0a70b-492b-4f1b-be45-6f9b2676c32a admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.478346 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0a70b-492b-4f1b-be45-6f9b2676c32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.478585 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0a70b-492b-4f1b-be45-6f9b2676c32a admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.479075 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0a70b-492b-4f1b-be45-6f9b2676c32a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:40:43.479370 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0a70b-492b-4f1b-be45-6f9b2676c32a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:43.481147 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0a70b-492b-4f1b-be45-6f9b2676c32a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:43.488943 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b4290-e5af-4461-b44b-30720a15994f None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:43.491005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a87b4290-e5af-4461-b44b-30720a15994f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:43.496921 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1421/2814] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:40:43.501954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0a70b-492b-4f1b-be45-6f9b2676c32a admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:43.509322 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b4290-e5af-4461-b44b-30720a15994f None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:43.514030 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0a70b-492b-4f1b-be45-6f9b2676c32a admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:43.516525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:43.516976 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.517342 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.517758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02 admin admin] PATH_INFO: `/v3/projects/7ecc6da53cc74bc78b8deeb628c3bbf9` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.519745 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ecc6da53cc74bc78b8deeb628c3bbf9)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:43.521752 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd31b4-b99a-4afd-b286-0c703bbedefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:43.522209 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:43.522530 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd31b4-b99a-4afd-b286-0c703bbedefd admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.522884 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd31b4-b99a-4afd-b286-0c703bbedefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.523383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd31b4-b99a-4afd-b286-0c703bbedefd admin admin] PATH_INFO: `/v3/projects` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.524710 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd31b4-b99a-4afd-b286-0c703bbedefd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:43.526902 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd31b4-b99a-4afd-b286-0c703bbedefd admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:43.577319 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1422/2815] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:43 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:43.581920 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd31b4-b99a-4afd-b286-0c703bbedefd admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:43.602639 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ecc6da53cc74bc78b8deeb628c3bbf9', 'request_id': 'req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:43.611129 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ecc6da53cc74bc78b8deeb628c3bbf9', 'request_id': 'req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:43.620278 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ecc6da53cc74bc78b8deeb628c3bbf9', 'request_id': 'req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:43.651570 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02 admin admin] The token cache is being invalidate because project 7ecc6da53cc74bc78b8deeb628c3bbf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:40:43.651991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:43.656472 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:40:43.658510 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1394/2816] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:40:43 2022] DELETE /identity/v3/projects/7ecc6da53cc74bc78b8deeb628c3bbf9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:43.680718 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd31b4-b99a-4afd-b286-0c703bbedefd admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:43.760168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293d9d2-fd90-453d-a42d-0bed828f401c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:43.760719 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293d9d2-fd90-453d-a42d-0bed828f401c admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.761011 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293d9d2-fd90-453d-a42d-0bed828f401c admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.761285 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293d9d2-fd90-453d-a42d-0bed828f401c admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.761956 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293d9d2-fd90-453d-a42d-0bed828f401c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:43.764154 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293d9d2-fd90-453d-a42d-0bed828f401c admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:43.791999 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1423/2817] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:43 2022] POST /identity/v3/users => generated 292 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:43.797490 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:43.799753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:43.815919 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f52bd9-3cbd-4a6a-b90b-e2a174a25c02 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:43.821285 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a95b61-cd91-49dd-9006-3e259a416bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:43.821923 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a95b61-cd91-49dd-9006-3e259a416bda admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.822203 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a95b61-cd91-49dd-9006-3e259a416bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.822476 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a95b61-cd91-49dd-9006-3e259a416bda admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.823246 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a95b61-cd91-49dd-9006-3e259a416bda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:43.825413 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a95b61-cd91-49dd-9006-3e259a416bda admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:43.838078 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1395/2818] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:43.842779 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293d9d2-fd90-453d-a42d-0bed828f401c admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:43.857263 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293d9d2-fd90-453d-a42d-0bed828f401c admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:43.863652 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4bc2a-a1e8-48e7-b1ee-c2ea016e9d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:43.866390 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4bc2a-a1e8-48e7-b1ee-c2ea016e9d1d admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.866815 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4bc2a-a1e8-48e7-b1ee-c2ea016e9d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.867158 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4bc2a-a1e8-48e7-b1ee-c2ea016e9d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.868791 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4bc2a-a1e8-48e7-b1ee-c2ea016e9d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:43.871866 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4bc2a-a1e8-48e7-b1ee-c2ea016e9d1d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:43.897643 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1424/2819] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:43.899084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a95b61-cd91-49dd-9006-3e259a416bda admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:43.913794 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a95b61-cd91-49dd-9006-3e259a416bda admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:43.924267 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8a89e-9dce-400b-a765-d9cb9d5865b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:43.925442 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8a89e-9dce-400b-a765-d9cb9d5865b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.926499 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8a89e-9dce-400b-a765-d9cb9d5865b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.926961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8a89e-9dce-400b-a765-d9cb9d5865b9 admin admin] PATH_INFO: `/v3/projects/399ad86f3c61455bba626fd9e556e6e0/users/8aedc727a81a40c2993097e35485233c/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.952764 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8a89e-9dce-400b-a765-d9cb9d5865b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=399ad86f3c61455bba626fd9e556e6e0, user_id=8aedc727a81a40c2993097e35485233c, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:43.955508 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8a89e-9dce-400b-a765-d9cb9d5865b9 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:43.968700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1396/2820] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:43 2022] PUT /identity/v3/projects/399ad86f3c61455bba626fd9e556e6e0/users/8aedc727a81a40c2993097e35485233c/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:43.973773 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799e874-b588-4350-a469-e04eb3d27c3a None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:43.974064 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799e874-b588-4350-a469-e04eb3d27c3a None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:43.974369 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799e874-b588-4350-a469-e04eb3d27c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:43.989827 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:43.995942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-6799e874-b588-4350-a469-e04eb3d27c3a None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:43.998399 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-6799e874-b588-4350-a469-e04eb3d27c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:44.059695 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1425/2821] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:44.069838 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8a89e-9dce-400b-a765-d9cb9d5865b9 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:44.071540 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-80d8a89e-9dce-400b-a765-d9cb9d5865b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:44.131633 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8a89e-9dce-400b-a765-d9cb9d5865b9 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:44.135441 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7dd9cd-58dd-4e0f-9c55-31576c116513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:44.136077 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7dd9cd-58dd-4e0f-9c55-31576c116513 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:44.136355 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7dd9cd-58dd-4e0f-9c55-31576c116513 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:44.136656 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7dd9cd-58dd-4e0f-9c55-31576c116513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:44.145756 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7dd9cd-58dd-4e0f-9c55-31576c116513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:44.147160 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7dd9cd-58dd-4e0f-9c55-31576c116513 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:44.156460 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1397/2822] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:48.975572 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe19964-0225-4fb5-a1a6-b1b8740d9a60 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:48.976487 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe19964-0225-4fb5-a1a6-b1b8740d9a60 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:48.976792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe19964-0225-4fb5-a1a6-b1b8740d9a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:49.021670 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-efe19964-0225-4fb5-a1a6-b1b8740d9a60 None None] MFA Rules not processed for user `8aedc727a81a40c2993097e35485233c`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:49.024116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-efe19964-0225-4fb5-a1a6-b1b8740d9a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:49.164723 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1426/2823] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:49.171047 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7dd9cd-58dd-4e0f-9c55-31576c116513 service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:49.176440 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:49.184674 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7dd9cd-58dd-4e0f-9c55-31576c116513 service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:49.191303 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e2da4-5112-43e7-a9b2-be2899a97023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:49.192023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e2da4-5112-43e7-a9b2-be2899a97023 admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:49.192298 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e2da4-5112-43e7-a9b2-be2899a97023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:49.192853 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e2da4-5112-43e7-a9b2-be2899a97023 admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:49.193835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e2da4-5112-43e7-a9b2-be2899a97023 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:49.196916 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e2da4-5112-43e7-a9b2-be2899a97023 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:49.251099 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1398/2824] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:40:49 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:40:49.252262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe19964-0225-4fb5-a1a6-b1b8740d9a60 None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:49.265118 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe19964-0225-4fb5-a1a6-b1b8740d9a60 None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:49.361899 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70602f-88b3-4ee0-9e27-6ff172eb8f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:49.362894 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70602f-88b3-4ee0-9e27-6ff172eb8f8e admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:49.363284 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70602f-88b3-4ee0-9e27-6ff172eb8f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:49.363692 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70602f-88b3-4ee0-9e27-6ff172eb8f8e admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:49.364449 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70602f-88b3-4ee0-9e27-6ff172eb8f8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:49.367665 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70602f-88b3-4ee0-9e27-6ff172eb8f8e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:49.397621 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1427/2825] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:40:49 2022] POST /identity/v3/users => generated 298 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 12:40:49.401748 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e2da4-5112-43e7-a9b2-be2899a97023 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:49.415178 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e2da4-5112-43e7-a9b2-be2899a97023 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:49.419950 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7d6ef-0e30-4d35-8f12-289dcf4d9bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:49.420753 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7d6ef-0e30-4d35-8f12-289dcf4d9bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:49.421044 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7d6ef-0e30-4d35-8f12-289dcf4d9bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:49.421321 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7d6ef-0e30-4d35-8f12-289dcf4d9bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:49.421968 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7d6ef-0e30-4d35-8f12-289dcf4d9bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:49.423569 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7d6ef-0e30-4d35-8f12-289dcf4d9bb5 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:49.434345 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1399/2826] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:49.439063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70602f-88b3-4ee0-9e27-6ff172eb8f8e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:49.449374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70602f-88b3-4ee0-9e27-6ff172eb8f8e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:49.454924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80b6ab-0949-4300-b72b-03ad4ef7bf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:49.455593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80b6ab-0949-4300-b72b-03ad4ef7bf07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:49.455911 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80b6ab-0949-4300-b72b-03ad4ef7bf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:49.456189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80b6ab-0949-4300-b72b-03ad4ef7bf07 admin admin] PATH_INFO: `/v3/projects/64c5f0c44e604538a9a7397d21a5e814/users/ad623d8db0654cda8f9937d33af849aa/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:49.476909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80b6ab-0949-4300-b72b-03ad4ef7bf07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c5f0c44e604538a9a7397d21a5e814, user_id=ad623d8db0654cda8f9937d33af849aa, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:49.480235 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80b6ab-0949-4300-b72b-03ad4ef7bf07 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:49.496063 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1428/2827] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:49 2022] PUT /identity/v3/projects/64c5f0c44e604538a9a7397d21a5e814/users/ad623d8db0654cda8f9937d33af849aa/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:40:49.501033 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7d6ef-0e30-4d35-8f12-289dcf4d9bb5 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:49.511924 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7d6ef-0e30-4d35-8f12-289dcf4d9bb5 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:49.571003 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36073c1d-d600-4c0e-a7e1-97eab47fee99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:49.571607 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36073c1d-d600-4c0e-a7e1-97eab47fee99 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:49.571919 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36073c1d-d600-4c0e-a7e1-97eab47fee99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:49.572180 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36073c1d-d600-4c0e-a7e1-97eab47fee99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:49.572851 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36073c1d-d600-4c0e-a7e1-97eab47fee99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:49.574698 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36073c1d-d600-4c0e-a7e1-97eab47fee99 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:49.585178 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1400/2828] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:40:49.589814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80b6ab-0949-4300-b72b-03ad4ef7bf07 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:49.597389 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:49.604841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80b6ab-0949-4300-b72b-03ad4ef7bf07 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:49.611772 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7ca27-6a4c-496d-a422-70bd67908b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:49.612527 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7ca27-6a4c-496d-a422-70bd67908b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:49.612919 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7ca27-6a4c-496d-a422-70bd67908b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:49.613230 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7ca27-6a4c-496d-a422-70bd67908b8e admin admin] PATH_INFO: `/v3/projects/64c5f0c44e604538a9a7397d21a5e814/users/ad623d8db0654cda8f9937d33af849aa/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:49.617290 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e7ca27-6a4c-496d-a422-70bd67908b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c5f0c44e604538a9a7397d21a5e814, user_id=ad623d8db0654cda8f9937d33af849aa, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:49.621041 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e7ca27-6a4c-496d-a422-70bd67908b8e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:49.639119 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1429/2829] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:40:49 2022] PUT /identity/v3/projects/64c5f0c44e604538a9a7397d21a5e814/users/ad623d8db0654cda8f9937d33af849aa/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:40:55.046452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1b50a-6241-4530-8d8e-de25ca337841 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:55.047112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1b50a-6241-4530-8d8e-de25ca337841 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:55.047112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c1b50a-6241-4530-8d8e-de25ca337841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:55.079169 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-78c1b50a-6241-4530-8d8e-de25ca337841 None None] MFA Rules not processed for user `ad623d8db0654cda8f9937d33af849aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:55.080428 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-78c1b50a-6241-4530-8d8e-de25ca337841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:55.149659 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1401/2830] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:40:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 12:40:55.162535 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7ca27-6a4c-496d-a422-70bd67908b8e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:55.163861 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-24e7ca27-6a4c-496d-a422-70bd67908b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:55.240684 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7ca27-6a4c-496d-a422-70bd67908b8e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:55.244435 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4186426-6a71-47ff-88bb-8e42ab724838 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd8df2f3357d4660bc22366f9746f89c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:55.245050 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4186426-6a71-47ff-88bb-8e42ab724838 service glance] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:55.245330 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4186426-6a71-47ff-88bb-8e42ab724838 service glance] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:55.245603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4186426-6a71-47ff-88bb-8e42ab724838 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:55.327247 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4186426-6a71-47ff-88bb-8e42ab724838 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:55.328882 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4186426-6a71-47ff-88bb-8e42ab724838 service glance] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:55.339172 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1430/2831] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:40:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:40:55.496566 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237b8b1-db9b-492a-8876-6efe13bed310 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:55.496856 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237b8b1-db9b-492a-8876-6efe13bed310 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:55.497349 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237b8b1-db9b-492a-8876-6efe13bed310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:55.538282 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-d237b8b1-db9b-492a-8876-6efe13bed310 None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:40:55.540895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-d237b8b1-db9b-492a-8876-6efe13bed310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:55.592144 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:40:55.627354 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1402/2832] 10.222.0.58 () {58 vars in 1009 bytes} [Wed Aug 24 12:40:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:40:55.715148 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4186426-6a71-47ff-88bb-8e42ab724838 service glance] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:40:55.717883 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f4186426-6a71-47ff-88bb-8e42ab724838 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:40:55.824598 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4186426-6a71-47ff-88bb-8e42ab724838 service glance] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:40:55.836547 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b46f40-07cf-4243-bd18-666912a24c3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:40:55.837189 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b46f40-07cf-4243-bd18-666912a24c3d service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:40:55.837473 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b46f40-07cf-4243-bd18-666912a24c3d service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:40:55.837757 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b46f40-07cf-4243-bd18-666912a24c3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:40:55.848693 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b46f40-07cf-4243-bd18-666912a24c3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:40:55.851234 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b46f40-07cf-4243-bd18-666912a24c3d service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:40:55.862169 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1431/2833] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:40:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 12:41:03.017428 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6b6c8-4c65-4910-81b3-026d3ac32c2f None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:03.017998 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6b6c8-4c65-4910-81b3-026d3ac32c2f None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:03.018111 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6b6c8-4c65-4910-81b3-026d3ac32c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:03.042290 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-e7e6b6c8-4c65-4910-81b3-026d3ac32c2f None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:03.044781 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-e7e6b6c8-4c65-4910-81b3-026d3ac32c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:03.052779 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1403/2834] 10.222.0.58 () {58 vars in 1009 bytes} [Wed Aug 24 12:41:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:03.072981 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b46f40-07cf-4243-bd18-666912a24c3d service swift] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:03.086493 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b46f40-07cf-4243-bd18-666912a24c3d service swift] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:03.092914 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709c29e-b183-4318-b0f0-06037059a4d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:03.093534 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709c29e-b183-4318-b0f0-06037059a4d3 service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:03.093819 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709c29e-b183-4318-b0f0-06037059a4d3 service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:03.094098 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709c29e-b183-4318-b0f0-06037059a4d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:03.107554 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709c29e-b183-4318-b0f0-06037059a4d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:03.109203 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709c29e-b183-4318-b0f0-06037059a4d3 service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:03.118740 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1432/2835] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:41:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 12:41:05.720902 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23dcc0-7c83-4523-8bb9-1e2a52bf4c88 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:05.721750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23dcc0-7c83-4523-8bb9-1e2a52bf4c88 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:05.721750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23dcc0-7c83-4523-8bb9-1e2a52bf4c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:05.742822 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-4c23dcc0-7c83-4523-8bb9-1e2a52bf4c88 None None] MFA Rules not processed for user `8aedc727a81a40c2993097e35485233c`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:05.744360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-4c23dcc0-7c83-4523-8bb9-1e2a52bf4c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:05.754898 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1404/2836] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:41:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:09.329477 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa4401-90e6-4b98-aa3e-ac494d734212 None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:09.330023 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa4401-90e6-4b98-aa3e-ac494d734212 None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:09.330255 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa4401-90e6-4b98-aa3e-ac494d734212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:09.353371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-f3aa4401-90e6-4b98-aa3e-ac494d734212 None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:09.354971 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f3aa4401-90e6-4b98-aa3e-ac494d734212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:09.367023 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1433/2837] 10.222.0.58 () {58 vars in 1009 bytes} [Wed Aug 24 12:41:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:09.425589 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23dcc0-7c83-4523-8bb9-1e2a52bf4c88 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:09.441412 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23dcc0-7c83-4523-8bb9-1e2a52bf4c88 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:09.456500 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6c39c-0dc0-48a0-994d-836088f6fa11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:09.457130 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6c39c-0dc0-48a0-994d-836088f6fa11 service swift] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:09.457409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6c39c-0dc0-48a0-994d-836088f6fa11 service swift] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:09.457683 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6c39c-0dc0-48a0-994d-836088f6fa11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:09.477798 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6c39c-0dc0-48a0-994d-836088f6fa11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:09.479416 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6c39c-0dc0-48a0-994d-836088f6fa11 service swift] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:09.496287 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1405/2838] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:41:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:41:19.347336 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=93,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:19.493045 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=21 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:20.967806 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bdc18-cf99-4527-adf3-1ee3ec65ddae None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:20.968292 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bdc18-cf99-4527-adf3-1ee3ec65ddae None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:20.968449 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bdc18-cf99-4527-adf3-1ee3ec65ddae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:20.990108 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-908bdc18-cf99-4527-adf3-1ee3ec65ddae None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:20.991458 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-908bdc18-cf99-4527-adf3-1ee3ec65ddae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:21.002129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1434/2839] 10.222.0.58 () {58 vars in 967 bytes} [Wed Aug 24 12:41:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:21.015094 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6c39c-0dc0-48a0-994d-836088f6fa11 service swift] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:21.026605 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6c39c-0dc0-48a0-994d-836088f6fa11 service swift] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:21.033464 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81cbee-afcf-4ae6-80df-7254ab655d53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:21.034218 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81cbee-afcf-4ae6-80df-7254ab655d53 service swift] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:21.034529 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81cbee-afcf-4ae6-80df-7254ab655d53 service swift] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:21.034869 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81cbee-afcf-4ae6-80df-7254ab655d53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:21.043835 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81cbee-afcf-4ae6-80df-7254ab655d53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:21.047939 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bdc18-cf99-4527-adf3-1ee3ec65ddae None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:21.048804 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81cbee-afcf-4ae6-80df-7254ab655d53 service swift] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:21.049814 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-908bdc18-cf99-4527-adf3-1ee3ec65ddae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:21.062469 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1406/2840] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:41:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:41:21.135582 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bdc18-cf99-4527-adf3-1ee3ec65ddae None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:21.140985 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e34b6-9dd5-4adc-a3da-c01af795288a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:21.141751 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e34b6-9dd5-4adc-a3da-c01af795288a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:21.142038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e34b6-9dd5-4adc-a3da-c01af795288a admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:21.142314 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e34b6-9dd5-4adc-a3da-c01af795288a admin admin] PATH_INFO: `/v3/users/9c40bc0027e94421a1da7b8016307b67` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:21.143505 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e34b6-9dd5-4adc-a3da-c01af795288a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c40bc0027e94421a1da7b8016307b67)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:21.145528 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e34b6-9dd5-4adc-a3da-c01af795288a admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:21.203374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da2e34b6-9dd5-4adc-a3da-c01af795288a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c40bc0027e94421a1da7b8016307b67'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:21.204342 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da2e34b6-9dd5-4adc-a3da-c01af795288a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c40bc0027e94421a1da7b8016307b67'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:21.214547 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da2e34b6-9dd5-4adc-a3da-c01af795288a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c40bc0027e94421a1da7b8016307b67'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:21.224431 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da2e34b6-9dd5-4adc-a3da-c01af795288a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c40bc0027e94421a1da7b8016307b67'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:21.240439 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-da2e34b6-9dd5-4adc-a3da-c01af795288a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c40bc0027e94421a1da7b8016307b67'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:21.247755 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1435/2841] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:41:21 2022] DELETE /identity/v3/users/9c40bc0027e94421a1da7b8016307b67 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:41:21.594969 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81cbee-afcf-4ae6-80df-7254ab655d53 service swift] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:21.607365 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81cbee-afcf-4ae6-80df-7254ab655d53 service swift] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:21.679795 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6044c-8157-4fdd-b534-1551cc93a43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:21.680465 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6044c-8157-4fdd-b534-1551cc93a43d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:21.680748 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6044c-8157-4fdd-b534-1551cc93a43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:21.681027 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6044c-8157-4fdd-b534-1551cc93a43d admin admin] PATH_INFO: `/v3/projects/474008375491499b85702c8824bd1981` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:21.682213 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6044c-8157-4fdd-b534-1551cc93a43d admin admin] RBAC: Authorizing `identity:delete_project(project_id=474008375491499b85702c8824bd1981)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:21.684741 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6044c-8157-4fdd-b534-1551cc93a43d admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:21.734396 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1aa6044c-8157-4fdd-b534-1551cc93a43d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '474008375491499b85702c8824bd1981', 'request_id': 'req-1aa6044c-8157-4fdd-b534-1551cc93a43d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:21.739780 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1aa6044c-8157-4fdd-b534-1551cc93a43d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '474008375491499b85702c8824bd1981', 'request_id': 'req-1aa6044c-8157-4fdd-b534-1551cc93a43d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:21.748084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1aa6044c-8157-4fdd-b534-1551cc93a43d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '474008375491499b85702c8824bd1981', 'request_id': 'req-1aa6044c-8157-4fdd-b534-1551cc93a43d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:21.758887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1aa6044c-8157-4fdd-b534-1551cc93a43d admin admin] The token cache is being invalidate because project 474008375491499b85702c8824bd1981 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:41:21.759170 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1aa6044c-8157-4fdd-b534-1551cc93a43d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:21.759814 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1aa6044c-8157-4fdd-b534-1551cc93a43d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:21.761283 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1407/2842] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:41:21 2022] DELETE /identity/v3/projects/474008375491499b85702c8824bd1981 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:41:21.766167 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a89f2-2a47-453d-a90a-2a4d840cc9ad None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:21.766484 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a89f2-2a47-453d-a90a-2a4d840cc9ad None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:21.766759 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a89f2-2a47-453d-a90a-2a4d840cc9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:21.788168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-980a89f2-2a47-453d-a90a-2a4d840cc9ad None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:21.789380 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-980a89f2-2a47-453d-a90a-2a4d840cc9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:21.850005 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1436/2843] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:21.859303 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa83114-e701-49ec-957f-0fb8c898e6b3 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:21.859812 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa83114-e701-49ec-957f-0fb8c898e6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:21.860163 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa83114-e701-49ec-957f-0fb8c898e6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:21.881652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-0aa83114-e701-49ec-957f-0fb8c898e6b3 None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:21.883009 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-0aa83114-e701-49ec-957f-0fb8c898e6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:21.891866 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1408/2844] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:21.896000 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a89f2-2a47-453d-a90a-2a4d840cc9ad None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:21.906440 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a89f2-2a47-453d-a90a-2a4d840cc9ad None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:21.911355 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711cae7-b8da-43f7-8f8d-6a08145eda5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:21.911981 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711cae7-b8da-43f7-8f8d-6a08145eda5d admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:21.912260 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711cae7-b8da-43f7-8f8d-6a08145eda5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:21.912555 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711cae7-b8da-43f7-8f8d-6a08145eda5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:21.913205 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711cae7-b8da-43f7-8f8d-6a08145eda5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 12:41:21.913505 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711cae7-b8da-43f7-8f8d-6a08145eda5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:21.915103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711cae7-b8da-43f7-8f8d-6a08145eda5d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:21.924656 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:21.928741 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1437/2845] 10.222.0.58 () {58 vars in 1153 bytes} [Wed Aug 24 12:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 12:41:21.933082 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa83114-e701-49ec-957f-0fb8c898e6b3 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:21.942639 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa83114-e701-49ec-957f-0fb8c898e6b3 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:21.947346 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42f0d4-1ffb-4e19-902e-61c57237c9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:21.947990 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42f0d4-1ffb-4e19-902e-61c57237c9ee admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:21.948272 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42f0d4-1ffb-4e19-902e-61c57237c9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:21.948814 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42f0d4-1ffb-4e19-902e-61c57237c9ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:21.949475 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42f0d4-1ffb-4e19-902e-61c57237c9ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:21.951376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f42f0d4-1ffb-4e19-902e-61c57237c9ee admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:21.981943 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1409/2846] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:41:21 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:41:21.983315 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711cae7-b8da-43f7-8f8d-6a08145eda5d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:21.993573 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711cae7-b8da-43f7-8f8d-6a08145eda5d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:22.049083 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59162a93-2b12-4f4d-892a-96ab5017033d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:22.049688 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59162a93-2b12-4f4d-892a-96ab5017033d admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:22.049971 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59162a93-2b12-4f4d-892a-96ab5017033d admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:22.050245 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59162a93-2b12-4f4d-892a-96ab5017033d admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:22.050862 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59162a93-2b12-4f4d-892a-96ab5017033d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:22.053004 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59162a93-2b12-4f4d-892a-96ab5017033d admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:22.074153 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1438/2847] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:41:21 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:41:22.078392 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42f0d4-1ffb-4e19-902e-61c57237c9ee admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:22.088849 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f42f0d4-1ffb-4e19-902e-61c57237c9ee admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:22.094195 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6deae8-2f62-4ba8-a3d0-431d7efc8a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:22.094797 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6deae8-2f62-4ba8-a3d0-431d7efc8a2b admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:22.095077 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6deae8-2f62-4ba8-a3d0-431d7efc8a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:22.095350 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6deae8-2f62-4ba8-a3d0-431d7efc8a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:22.096027 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6deae8-2f62-4ba8-a3d0-431d7efc8a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:22.097614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6deae8-2f62-4ba8-a3d0-431d7efc8a2b admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:22.108452 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1410/2848] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:41:22.112329 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59162a93-2b12-4f4d-892a-96ab5017033d admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:22.122447 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59162a93-2b12-4f4d-892a-96ab5017033d admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:22.127139 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2f2a4b-305c-47ac-8858-0f6f46c7abf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:22.127772 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f2a4b-305c-47ac-8858-0f6f46c7abf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:22.128057 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f2a4b-305c-47ac-8858-0f6f46c7abf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:22.128335 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2f2a4b-305c-47ac-8858-0f6f46c7abf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:22.128975 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f2a4b-305c-47ac-8858-0f6f46c7abf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:22.130568 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2f2a4b-305c-47ac-8858-0f6f46c7abf7 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:22.141467 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1439/2849] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:41:22.145370 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6deae8-2f62-4ba8-a3d0-431d7efc8a2b admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:22.155891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6deae8-2f62-4ba8-a3d0-431d7efc8a2b admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:22.160400 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a7126-666b-46d7-87e7-b234206a82f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:22.161112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a7126-666b-46d7-87e7-b234206a82f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:22.161337 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a7126-666b-46d7-87e7-b234206a82f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:22.161621 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a7126-666b-46d7-87e7-b234206a82f9 admin admin] PATH_INFO: `/v3/projects/dcf72fd7c24f4691b49b142f53d81ea0/users/bf01094f0a444f2396426866d134d222/roles/156b5bb1275a4dbeb61dbdade2c20d21` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:22.176357 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a7126-666b-46d7-87e7-b234206a82f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcf72fd7c24f4691b49b142f53d81ea0, user_id=bf01094f0a444f2396426866d134d222, role_id=156b5bb1275a4dbeb61dbdade2c20d21)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:22.178995 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a7126-666b-46d7-87e7-b234206a82f9 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:22.191323 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1411/2850] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:41:22 2022] PUT /identity/v3/projects/dcf72fd7c24f4691b49b142f53d81ea0/users/bf01094f0a444f2396426866d134d222/roles/156b5bb1275a4dbeb61dbdade2c20d21 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:41:22.196622 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948f6ad-ad0d-4b09-8a05-9a300eaa637e None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:22.196911 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948f6ad-ad0d-4b09-8a05-9a300eaa637e None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:22.197223 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948f6ad-ad0d-4b09-8a05-9a300eaa637e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:22.217241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-1948f6ad-ad0d-4b09-8a05-9a300eaa637e None None] MFA Rules not processed for user `ec5a3f02319e4ee5bdd1b0a6b24084c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:22.218483 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1948f6ad-ad0d-4b09-8a05-9a300eaa637e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:22.276507 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1440/2851] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:22.286839 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a7126-666b-46d7-87e7-b234206a82f9 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:22.288582 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a38a7126-666b-46d7-87e7-b234206a82f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:22.306426 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:22.374793 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a7126-666b-46d7-87e7-b234206a82f9 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:22.378427 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b243b-f739-43cf-aa3f-a98b012c91a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:22.379033 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b243b-f739-43cf-aa3f-a98b012c91a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:22.379314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b243b-f739-43cf-aa3f-a98b012c91a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:22.379590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b243b-f739-43cf-aa3f-a98b012c91a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:22.389525 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b243b-f739-43cf-aa3f-a98b012c91a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:22.390983 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b243b-f739-43cf-aa3f-a98b012c91a9 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:22.400242 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1412/2852] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:41:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:41:23.096323 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948f6ad-ad0d-4b09-8a05-9a300eaa637e None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:23.098523 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-1948f6ad-ad0d-4b09-8a05-9a300eaa637e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:23.114694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948f6ad-ad0d-4b09-8a05-9a300eaa637e None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:23.121149 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f13f7b-1847-481c-b78a-24490cffc218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:23.122356 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f13f7b-1847-481c-b78a-24490cffc218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:23.123910 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f13f7b-1847-481c-b78a-24490cffc218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:23.124221 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f13f7b-1847-481c-b78a-24490cffc218 admin admin] PATH_INFO: `/v3/users/25fad82cbf944d7db13d6bf33d3297a1` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:23.126720 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f13f7b-1847-481c-b78a-24490cffc218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25fad82cbf944d7db13d6bf33d3297a1)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:23.131863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f13f7b-1847-481c-b78a-24490cffc218 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:23.191305 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97f13f7b-1847-481c-b78a-24490cffc218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25fad82cbf944d7db13d6bf33d3297a1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.192057 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97f13f7b-1847-481c-b78a-24490cffc218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25fad82cbf944d7db13d6bf33d3297a1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.201313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97f13f7b-1847-481c-b78a-24490cffc218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25fad82cbf944d7db13d6bf33d3297a1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.209546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97f13f7b-1847-481c-b78a-24490cffc218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25fad82cbf944d7db13d6bf33d3297a1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.218924 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:23.224364 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-97f13f7b-1847-481c-b78a-24490cffc218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25fad82cbf944d7db13d6bf33d3297a1'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.232497 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1441/2853] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:41:23 2022] DELETE /identity/v3/users/25fad82cbf944d7db13d6bf33d3297a1 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:41:23.443522 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b243b-f739-43cf-aa3f-a98b012c91a9 service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:23.464891 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b243b-f739-43cf-aa3f-a98b012c91a9 service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:23.550772 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:23.551417 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:23.551732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:23.552027 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74 admin admin] PATH_INFO: `/v3/projects/a6ce654b8dc4435981cb395c4b887ae7` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:23.553252 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ce654b8dc4435981cb395c4b887ae7)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:23.555173 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:23.619394 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ce654b8dc4435981cb395c4b887ae7', 'request_id': 'req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.628409 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ce654b8dc4435981cb395c4b887ae7', 'request_id': 'req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.640784 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ce654b8dc4435981cb395c4b887ae7', 'request_id': 'req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.656057 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74 admin admin] The token cache is being invalidate because project a6ce654b8dc4435981cb395c4b887ae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:41:23.656497 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.657258 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:41:23.658909 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1413/2854] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:41:23 2022] DELETE /identity/v3/projects/a6ce654b8dc4435981cb395c4b887ae7 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:41:26.884507 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a543b8-1c5f-4ab1-8f43-4f0a1b0dad9b None None] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:26.884991 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a543b8-1c5f-4ab1-8f43-4f0a1b0dad9b None None] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:26.885081 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a543b8-1c5f-4ab1-8f43-4f0a1b0dad9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:26.918412 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.auth.core [None req-92a543b8-1c5f-4ab1-8f43-4f0a1b0dad9b None None] MFA Rules not processed for user `bf01094f0a444f2396426866d134d222`. Rule list: `[]` (Enabled: `True`). {{(pid=81638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:26.920385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-92a543b8-1c5f-4ab1-8f43-4f0a1b0dad9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:26.991646 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1442/2855] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:41:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:26.996828 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:26.998461 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:27.029690 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:27.072997 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07b1cf-9d48-4d75-afb2-cdfd2720da74 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:27.077925 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6f059-6fa0-4088-bab3-bb06fae7286a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:27.078541 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6f059-6fa0-4088-bab3-bb06fae7286a admin admin] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:27.078823 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6f059-6fa0-4088-bab3-bb06fae7286a admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:27.079099 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6f059-6fa0-4088-bab3-bb06fae7286a admin admin] PATH_INFO: `/v3/projects` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:27.079763 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6f059-6fa0-4088-bab3-bb06fae7286a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:27.081865 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6f059-6fa0-4088-bab3-bb06fae7286a admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:27.111489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1414/2856] 10.222.0.58 () {60 vars in 1153 bytes} [Wed Aug 24 12:41:26 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:41:27.115557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a543b8-1c5f-4ab1-8f43-4f0a1b0dad9b None None] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:27.126312 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a543b8-1c5f-4ab1-8f43-4f0a1b0dad9b None None] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:27.184032 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab20bd5-5b50-4875-a85f-c1af366779ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:27.184637 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab20bd5-5b50-4875-a85f-c1af366779ea admin admin] REQUEST_METHOD: `POST` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:27.184912 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab20bd5-5b50-4875-a85f-c1af366779ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:27.185179 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab20bd5-5b50-4875-a85f-c1af366779ea admin admin] PATH_INFO: `/v3/users` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:27.185792 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab20bd5-5b50-4875-a85f-c1af366779ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:27.187812 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab20bd5-5b50-4875-a85f-c1af366779ea admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:27.209706 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1443/2857] 10.222.0.58 () {60 vars in 1144 bytes} [Wed Aug 24 12:41:27 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 12:41:27.213700 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6f059-6fa0-4088-bab3-bb06fae7286a admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:27.224433 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6f059-6fa0-4088-bab3-bb06fae7286a admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:27.229026 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73a375-45c8-4785-94d6-5cd61b27aea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:27.229617 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73a375-45c8-4785-94d6-5cd61b27aea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:27.229897 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73a375-45c8-4785-94d6-5cd61b27aea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:27.230172 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73a375-45c8-4785-94d6-5cd61b27aea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:27.231420 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73a375-45c8-4785-94d6-5cd61b27aea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:27.233232 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73a375-45c8-4785-94d6-5cd61b27aea1 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:27.243269 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1415/2858] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 12:41:27.247426 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab20bd5-5b50-4875-a85f-c1af366779ea admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:27.257313 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab20bd5-5b50-4875-a85f-c1af366779ea admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:27.262020 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcabcc-1b7f-40d8-a0f0-6266eb6718b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:27.262680 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcabcc-1b7f-40d8-a0f0-6266eb6718b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:27.262957 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcabcc-1b7f-40d8-a0f0-6266eb6718b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:27.263282 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbcabcc-1b7f-40d8-a0f0-6266eb6718b4 admin admin] PATH_INFO: `/v3/projects/b41760d63ef94abf96a0b955325cdf56/users/e908095d089d4729b22a5cfb3f9d9c0b/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:27.278584 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbcabcc-1b7f-40d8-a0f0-6266eb6718b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b41760d63ef94abf96a0b955325cdf56, user_id=e908095d089d4729b22a5cfb3f9d9c0b, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:27.281356 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbcabcc-1b7f-40d8-a0f0-6266eb6718b4 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:27.295273 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1444/2859] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:41:27 2022] PUT /identity/v3/projects/b41760d63ef94abf96a0b955325cdf56/users/e908095d089d4729b22a5cfb3f9d9c0b/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:41:27.299275 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73a375-45c8-4785-94d6-5cd61b27aea1 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:27.310043 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73a375-45c8-4785-94d6-5cd61b27aea1 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:27.378672 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4cbd6-ba6c-4e4f-be8f-8bf96a71091f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:27.379286 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4cbd6-ba6c-4e4f-be8f-8bf96a71091f admin admin] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:27.379568 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4cbd6-ba6c-4e4f-be8f-8bf96a71091f admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:27.379880 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4cbd6-ba6c-4e4f-be8f-8bf96a71091f admin admin] PATH_INFO: `/v3/roles` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:27.380533 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4cbd6-ba6c-4e4f-be8f-8bf96a71091f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:27.382150 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4cbd6-ba6c-4e4f-be8f-8bf96a71091f admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:27.393500 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1416/2860] 10.222.0.58 () {58 vars in 1122 bytes} [Wed Aug 24 12:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 12:41:27.397901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcabcc-1b7f-40d8-a0f0-6266eb6718b4 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:27.409115 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbcabcc-1b7f-40d8-a0f0-6266eb6718b4 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:27.413782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec7037-e653-4a2e-af01-0ad5366e3a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:27.414504 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec7037-e653-4a2e-af01-0ad5366e3a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:27.415404 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec7037-e653-4a2e-af01-0ad5366e3a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:27.415793 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec7037-e653-4a2e-af01-0ad5366e3a66 admin admin] PATH_INFO: `/v3/projects/b41760d63ef94abf96a0b955325cdf56/users/e908095d089d4729b22a5cfb3f9d9c0b/roles/6f69d065257c40daa87e7e1e460a17d9` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:27.418411 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec7037-e653-4a2e-af01-0ad5366e3a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b41760d63ef94abf96a0b955325cdf56, user_id=e908095d089d4729b22a5cfb3f9d9c0b, role_id=6f69d065257c40daa87e7e1e460a17d9)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:27.421103 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec7037-e653-4a2e-af01-0ad5366e3a66 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:27.431316 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1445/2861] 10.222.0.58 () {60 vars in 1483 bytes} [Wed Aug 24 12:41:27 2022] PUT /identity/v3/projects/b41760d63ef94abf96a0b955325cdf56/users/e908095d089d4729b22a5cfb3f9d9c0b/roles/6f69d065257c40daa87e7e1e460a17d9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:41:32.276674 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8e2c8-fedb-49e6-ad57-361413e4a196 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:32.278253 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8e2c8-fedb-49e6-ad57-361413e4a196 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:32.278533 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8e2c8-fedb-49e6-ad57-361413e4a196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:32.322590 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-46c8e2c8-fedb-49e6-ad57-361413e4a196 None None] MFA Rules not processed for user `e908095d089d4729b22a5cfb3f9d9c0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:32.340579 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-46c8e2c8-fedb-49e6-ad57-361413e4a196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:32.453230 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1417/2862] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:41:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:32.467375 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec7037-e653-4a2e-af01-0ad5366e3a66 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:32.469353 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-27ec7037-e653-4a2e-af01-0ad5366e3a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:32.514617 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:32.584818 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec7037-e653-4a2e-af01-0ad5366e3a66 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:32.590863 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a3082-55b2-4459-9a23-c6642f35f656 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:32.591603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a3082-55b2-4459-9a23-c6642f35f656 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:32.592010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a3082-55b2-4459-9a23-c6642f35f656 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:32.592269 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a3082-55b2-4459-9a23-c6642f35f656 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:32.653619 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a3082-55b2-4459-9a23-c6642f35f656 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:32.655249 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a3082-55b2-4459-9a23-c6642f35f656 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:32.664954 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1446/2863] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:41:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:41:33.685338 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdf349-f50b-4195-be94-b933e1a8744a None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:33.685915 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdf349-f50b-4195-be94-b933e1a8744a None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:33.685915 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdf349-f50b-4195-be94-b933e1a8744a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:33.711759 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-94bdf349-f50b-4195-be94-b933e1a8744a None None] MFA Rules not processed for user `bf01094f0a444f2396426866d134d222`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:41:33.713137 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-94bdf349-f50b-4195-be94-b933e1a8744a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:33.720583 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1418/2864] 10.222.0.58 () {58 vars in 958 bytes} [Wed Aug 24 12:41:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:41:38.741620 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a3082-55b2-4459-9a23-c6642f35f656 service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:41:38.743043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b69a3082-55b2-4459-9a23-c6642f35f656 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:38.807129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a3082-55b2-4459-9a23-c6642f35f656 service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:41:38.811002 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f79ff3-3d43-40f9-b119-035d82f40fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:41:38.811667 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f79ff3-3d43-40f9-b119-035d82f40fdf service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:41:38.811983 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f79ff3-3d43-40f9-b119-035d82f40fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:41:38.812259 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f79ff3-3d43-40f9-b119-035d82f40fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:41:38.814025 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b6f79ff3-3d43-40f9-b119-035d82f40fdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:41:38.882614 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f79ff3-3d43-40f9-b119-035d82f40fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:41:38.884297 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f79ff3-3d43-40f9-b119-035d82f40fdf service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:41:38.891043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:41:38.897196 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1447/2865] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:41:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:41:43.705758 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:04.459615 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdf349-f50b-4195-be94-b933e1a8744a None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:04.470213 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdf349-f50b-4195-be94-b933e1a8744a None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:04.473750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821da99-1ec9-4167-b4f5-4602e4e8240b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:04.474348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821da99-1ec9-4167-b4f5-4602e4e8240b service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:04.474630 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821da99-1ec9-4167-b4f5-4602e4e8240b service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:04.474904 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f821da99-1ec9-4167-b4f5-4602e4e8240b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:04.484663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821da99-1ec9-4167-b4f5-4602e4e8240b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:04.486096 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f821da99-1ec9-4167-b4f5-4602e4e8240b service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:04.495558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1419/2866] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:42:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:42:14.491239 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=6 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:23.268022 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f79ff3-3d43-40f9-b119-035d82f40fdf service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:23.269511 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-b6f79ff3-3d43-40f9-b119-035d82f40fdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:23.281359 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f79ff3-3d43-40f9-b119-035d82f40fdf service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:23.284964 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678fed0-5777-4fd8-a14b-34425165d64a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:23.285560 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678fed0-5777-4fd8-a14b-34425165d64a service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:23.285841 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678fed0-5777-4fd8-a14b-34425165d64a service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:23.286116 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678fed0-5777-4fd8-a14b-34425165d64a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:23.287800 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-c678fed0-5777-4fd8-a14b-34425165d64a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:23.300881 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678fed0-5777-4fd8-a14b-34425165d64a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:23.302302 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678fed0-5777-4fd8-a14b-34425165d64a service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:23.313129 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1448/2867] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:42:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:42:24.195423 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821da99-1ec9-4167-b4f5-4602e4e8240b service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:24.205558 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f821da99-1ec9-4167-b4f5-4602e4e8240b service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:24.209531 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542990df-4b1a-4ff7-ba99-8c334c055314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:24.210129 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542990df-4b1a-4ff7-ba99-8c334c055314 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:24.210404 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542990df-4b1a-4ff7-ba99-8c334c055314 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:24.210669 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542990df-4b1a-4ff7-ba99-8c334c055314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:24.212484 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-542990df-4b1a-4ff7-ba99-8c334c055314 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:24.275028 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542990df-4b1a-4ff7-ba99-8c334c055314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:24.277137 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542990df-4b1a-4ff7-ba99-8c334c055314 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:24.286750 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1420/2868] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:42:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:42:25.776191 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678fed0-5777-4fd8-a14b-34425165d64a service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:25.784901 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678fed0-5777-4fd8-a14b-34425165d64a service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:25.788593 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c0e0b-2d41-45b8-9990-6e8ccbd7c462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:25.789182 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c0e0b-2d41-45b8-9990-6e8ccbd7c462 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:25.789464 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c0e0b-2d41-45b8-9990-6e8ccbd7c462 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:25.789732 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c0e0b-2d41-45b8-9990-6e8ccbd7c462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:25.791441 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-996c0e0b-2d41-45b8-9990-6e8ccbd7c462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:25.805660 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c0e0b-2d41-45b8-9990-6e8ccbd7c462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:25.807085 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c0e0b-2d41-45b8-9990-6e8ccbd7c462 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:25.819345 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1449/2869] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:42:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:42:30.906158 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542990df-4b1a-4ff7-ba99-8c334c055314 service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:30.908381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-542990df-4b1a-4ff7-ba99-8c334c055314 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:30.922136 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542990df-4b1a-4ff7-ba99-8c334c055314 service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:30.926793 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49671852-3fef-4bbd-92f9-994f2179f8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:30.927426 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49671852-3fef-4bbd-92f9-994f2179f8ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:30.927740 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49671852-3fef-4bbd-92f9-994f2179f8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:30.928063 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49671852-3fef-4bbd-92f9-994f2179f8ca admin admin] PATH_INFO: `/v3/users/18f8f419b523440f8a4ca06dc83bff40` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:30.929240 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49671852-3fef-4bbd-92f9-994f2179f8ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=18f8f419b523440f8a4ca06dc83bff40)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:30.931217 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49671852-3fef-4bbd-92f9-994f2179f8ca admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:30.979164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-49671852-3fef-4bbd-92f9-994f2179f8ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18f8f419b523440f8a4ca06dc83bff40'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:30.980422 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-49671852-3fef-4bbd-92f9-994f2179f8ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18f8f419b523440f8a4ca06dc83bff40'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:30.981023 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-49671852-3fef-4bbd-92f9-994f2179f8ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18f8f419b523440f8a4ca06dc83bff40'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:30.988119 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-49671852-3fef-4bbd-92f9-994f2179f8ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18f8f419b523440f8a4ca06dc83bff40'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:30.997058 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-49671852-3fef-4bbd-92f9-994f2179f8ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18f8f419b523440f8a4ca06dc83bff40'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:31.016220 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1421/2870] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:42:30 2022] DELETE /identity/v3/users/18f8f419b523440f8a4ca06dc83bff40 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:42:31.188383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c0e0b-2d41-45b8-9990-6e8ccbd7c462 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:31.199872 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c0e0b-2d41-45b8-9990-6e8ccbd7c462 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:31.257980 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728813ef-ec4a-4641-8bdf-eb1c05a74be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:31.258609 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728813ef-ec4a-4641-8bdf-eb1c05a74be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:31.258893 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728813ef-ec4a-4641-8bdf-eb1c05a74be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:31.259168 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728813ef-ec4a-4641-8bdf-eb1c05a74be8 admin admin] PATH_INFO: `/v3/projects/aee96e55a6a34dbb81847961d87abad0` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:31.260459 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728813ef-ec4a-4641-8bdf-eb1c05a74be8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aee96e55a6a34dbb81847961d87abad0)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:31.262374 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728813ef-ec4a-4641-8bdf-eb1c05a74be8 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:31.308359 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-728813ef-ec4a-4641-8bdf-eb1c05a74be8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aee96e55a6a34dbb81847961d87abad0', 'request_id': 'req-728813ef-ec4a-4641-8bdf-eb1c05a74be8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:31.313836 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-728813ef-ec4a-4641-8bdf-eb1c05a74be8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aee96e55a6a34dbb81847961d87abad0', 'request_id': 'req-728813ef-ec4a-4641-8bdf-eb1c05a74be8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:31.321967 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-728813ef-ec4a-4641-8bdf-eb1c05a74be8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aee96e55a6a34dbb81847961d87abad0', 'request_id': 'req-728813ef-ec4a-4641-8bdf-eb1c05a74be8'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:31.333569 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-728813ef-ec4a-4641-8bdf-eb1c05a74be8 admin admin] The token cache is being invalidate because project aee96e55a6a34dbb81847961d87abad0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:42:31.333861 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-728813ef-ec4a-4641-8bdf-eb1c05a74be8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:31.334474 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-728813ef-ec4a-4641-8bdf-eb1c05a74be8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:31.336005 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1450/2871] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:42:31 2022] DELETE /identity/v3/projects/aee96e55a6a34dbb81847961d87abad0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:42:31.340598 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49671852-3fef-4bbd-92f9-994f2179f8ca admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:31.341967 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-49671852-3fef-4bbd-92f9-994f2179f8ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:31.355407 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49671852-3fef-4bbd-92f9-994f2179f8ca admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:31.411585 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0b47a-1f14-4a15-9b56-443ad94ef6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:31.412190 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0b47a-1f14-4a15-9b56-443ad94ef6eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:31.412455 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0b47a-1f14-4a15-9b56-443ad94ef6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:31.412732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0b47a-1f14-4a15-9b56-443ad94ef6eb admin admin] PATH_INFO: `/v3/users/e3df898326404ea4b1343fbcb5d70a9d` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:31.413855 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0b47a-1f14-4a15-9b56-443ad94ef6eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3df898326404ea4b1343fbcb5d70a9d)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:31.415807 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea0b47a-1f14-4a15-9b56-443ad94ef6eb admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:31.421103 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:31.462010 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-4ea0b47a-1f14-4a15-9b56-443ad94ef6eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3df898326404ea4b1343fbcb5d70a9d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:31.462653 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-4ea0b47a-1f14-4a15-9b56-443ad94ef6eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3df898326404ea4b1343fbcb5d70a9d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:31.463250 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-4ea0b47a-1f14-4a15-9b56-443ad94ef6eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3df898326404ea4b1343fbcb5d70a9d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:31.470732 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-4ea0b47a-1f14-4a15-9b56-443ad94ef6eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3df898326404ea4b1343fbcb5d70a9d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:31.478172 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-4ea0b47a-1f14-4a15-9b56-443ad94ef6eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3df898326404ea4b1343fbcb5d70a9d'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:31.495844 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1422/2872] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:42:31 2022] DELETE /identity/v3/users/e3df898326404ea4b1343fbcb5d70a9d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:42:31.662995 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728813ef-ec4a-4641-8bdf-eb1c05a74be8 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:42:31.664453 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-728813ef-ec4a-4641-8bdf-eb1c05a74be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:42:31.680581 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728813ef-ec4a-4641-8bdf-eb1c05a74be8 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:42:31.750315 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:42:31.751025 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:42:31.751222 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:42:31.751501 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f admin admin] PATH_INFO: `/v3/projects/159cb10e789d47ba9c4f865c05e07491` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:42:31.752649 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f admin admin] RBAC: Authorizing `identity:delete_project(project_id=159cb10e789d47ba9c4f865c05e07491)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:42:31.754752 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:42:31.761662 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:31.804014 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '159cb10e789d47ba9c4f865c05e07491', 'request_id': 'req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:31.810245 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '159cb10e789d47ba9c4f865c05e07491', 'request_id': 'req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:31.817463 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '159cb10e789d47ba9c4f865c05e07491', 'request_id': 'req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:31.828857 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f admin admin] The token cache is being invalidate because project 159cb10e789d47ba9c4f865c05e07491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:42:31.829159 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:31.829769 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:42:31.831253 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1451/2873] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:42:31 2022] DELETE /identity/v3/projects/159cb10e789d47ba9c4f865c05e07491 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:42:41.491635 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:42:41.825884 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:43:03.136281 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0b47a-1f14-4a15-9b56-443ad94ef6eb admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:03.138030 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-4ea0b47a-1f14-4a15-9b56-443ad94ef6eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:03.210421 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea0b47a-1f14-4a15-9b56-443ad94ef6eb admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:03.289854 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e420c-afdf-4326-af1d-66170f9506c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:03.293395 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e420c-afdf-4326-af1d-66170f9506c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:03.294028 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e420c-afdf-4326-af1d-66170f9506c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:03.294360 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e420c-afdf-4326-af1d-66170f9506c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:03.300147 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-1f4e420c-afdf-4326-af1d-66170f9506c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:03.312451 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e420c-afdf-4326-af1d-66170f9506c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:03.314377 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e420c-afdf-4326-af1d-66170f9506c1 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:03.325314 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1423/2874] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:43:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:43:09.429360 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:09.432276 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:09.513054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e4f94-5ded-4f21-b233-d0c4eb6ed02f admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:09.518896 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d969e8-d62a-4921-960e-d5cf822666db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:09.519553 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d969e8-d62a-4921-960e-d5cf822666db service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:09.519867 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d969e8-d62a-4921-960e-d5cf822666db service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:09.520392 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d969e8-d62a-4921-960e-d5cf822666db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:09.522200 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-71d969e8-d62a-4921-960e-d5cf822666db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:09.535974 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d969e8-d62a-4921-960e-d5cf822666db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:09.537596 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d969e8-d62a-4921-960e-d5cf822666db service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:09.549351 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1452/2875] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:43:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:43:10.885897 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefa4d0-ef4f-47ef-86b9-d0a9a2e86507 None None] REQUEST_METHOD: `POST` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:10.886384 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefa4d0-ef4f-47ef-86b9-d0a9a2e86507 None None] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:10.886488 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefa4d0-ef4f-47ef-86b9-d0a9a2e86507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:10.936654 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.auth.core [None req-feefa4d0-ef4f-47ef-86b9-d0a9a2e86507 None None] MFA Rules not processed for user `adc6a2b3f6bd45178cc7ffb181c3c016`. Rule list: `[]` (Enabled: `True`). {{(pid=81637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 12:43:10.937851 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-feefa4d0-ef4f-47ef-86b9-d0a9a2e86507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:11.002460 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1424/2876] 10.222.0.58 () {58 vars in 967 bytes} [Wed Aug 24 12:43:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 12:43:11.049010 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d969e8-d62a-4921-960e-d5cf822666db service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:11.050403 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-71d969e8-d62a-4921-960e-d5cf822666db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:11.144106 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d969e8-d62a-4921-960e-d5cf822666db service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:11.149100 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5271e27-2cb0-4ae6-a4b3-b2fcba50d862 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67756e31aeee48a897cfa8bd35a10119', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:11.150227 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5271e27-2cb0-4ae6-a4b3-b2fcba50d862 service swift] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:11.150506 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5271e27-2cb0-4ae6-a4b3-b2fcba50d862 service swift] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:11.150782 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5271e27-2cb0-4ae6-a4b3-b2fcba50d862 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:11.158734 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5271e27-2cb0-4ae6-a4b3-b2fcba50d862 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:11.160357 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5271e27-2cb0-4ae6-a4b3-b2fcba50d862 service swift] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:11.168822 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1453/2877] 10.222.0.58 () {60 vars in 1384 bytes} [Wed Aug 24 12:43:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 12:43:17.783542 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feefa4d0-ef4f-47ef-86b9-d0a9a2e86507 None None] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:17.785161 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-feefa4d0-ef4f-47ef-86b9-d0a9a2e86507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:17.826266 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=100 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:43:17.859511 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feefa4d0-ef4f-47ef-86b9-d0a9a2e86507 None None] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:17.863976 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486e9db-5be5-418a-b722-b7df346c40e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:17.864614 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486e9db-5be5-418a-b722-b7df346c40e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:17.864890 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486e9db-5be5-418a-b722-b7df346c40e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:17.865164 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486e9db-5be5-418a-b722-b7df346c40e0 admin admin] PATH_INFO: `/v3/users/8aedc727a81a40c2993097e35485233c` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:17.866292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486e9db-5be5-418a-b722-b7df346c40e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aedc727a81a40c2993097e35485233c)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:17.868788 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5486e9db-5be5-418a-b722-b7df346c40e0 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:17.915439 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-5486e9db-5be5-418a-b722-b7df346c40e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aedc727a81a40c2993097e35485233c'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:17.916142 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-5486e9db-5be5-418a-b722-b7df346c40e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aedc727a81a40c2993097e35485233c'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:17.916748 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-5486e9db-5be5-418a-b722-b7df346c40e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aedc727a81a40c2993097e35485233c'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:17.923749 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-5486e9db-5be5-418a-b722-b7df346c40e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aedc727a81a40c2993097e35485233c'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:17.933573 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-5486e9db-5be5-418a-b722-b7df346c40e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aedc727a81a40c2993097e35485233c'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:17.951411 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1425/2878] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:43:17 2022] DELETE /identity/v3/users/8aedc727a81a40c2993097e35485233c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:43:18.127522 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5271e27-2cb0-4ae6-a4b3-b2fcba50d862 service swift] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:18.145275 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5271e27-2cb0-4ae6-a4b3-b2fcba50d862 service swift] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:18.197518 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=100 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:43:18.204952 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:18.205575 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:18.205877 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:18.206148 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e admin admin] PATH_INFO: `/v3/projects/399ad86f3c61455bba626fd9e556e6e0` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:18.207241 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e admin admin] RBAC: Authorizing `identity:delete_project(project_id=399ad86f3c61455bba626fd9e556e6e0)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:18.209196 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:18.254694 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399ad86f3c61455bba626fd9e556e6e0', 'request_id': 'req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:18.260600 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399ad86f3c61455bba626fd9e556e6e0', 'request_id': 'req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:18.267579 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399ad86f3c61455bba626fd9e556e6e0', 'request_id': 'req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:18.278883 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e admin admin] The token cache is being invalidate because project 399ad86f3c61455bba626fd9e556e6e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:43:18.279169 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:18.279823 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:18.281275 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1454/2879] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:43:18 2022] DELETE /identity/v3/projects/399ad86f3c61455bba626fd9e556e6e0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:43:18.285487 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486e9db-5be5-418a-b722-b7df346c40e0 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:18.286895 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-5486e9db-5be5-418a-b722-b7df346c40e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:18.300005 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486e9db-5be5-418a-b722-b7df346c40e0 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:18.356818 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab6ed9-4d6b-42b7-b010-a53d995ad240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:18.357347 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab6ed9-4d6b-42b7-b010-a53d995ad240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:18.357624 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab6ed9-4d6b-42b7-b010-a53d995ad240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:18.357927 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab6ed9-4d6b-42b7-b010-a53d995ad240 admin admin] PATH_INFO: `/v3/users/ad623d8db0654cda8f9937d33af849aa` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:18.359036 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab6ed9-4d6b-42b7-b010-a53d995ad240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad623d8db0654cda8f9937d33af849aa)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:18.360976 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab6ed9-4d6b-42b7-b010-a53d995ad240 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:18.406663 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-11ab6ed9-4d6b-42b7-b010-a53d995ad240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad623d8db0654cda8f9937d33af849aa'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:18.407329 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-11ab6ed9-4d6b-42b7-b010-a53d995ad240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad623d8db0654cda8f9937d33af849aa'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:18.408112 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-11ab6ed9-4d6b-42b7-b010-a53d995ad240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad623d8db0654cda8f9937d33af849aa'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:18.412945 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:43:18.416391 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-11ab6ed9-4d6b-42b7-b010-a53d995ad240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad623d8db0654cda8f9937d33af849aa'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:18.424376 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-11ab6ed9-4d6b-42b7-b010-a53d995ad240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad623d8db0654cda8f9937d33af849aa'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:18.442152 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1426/2880] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:43:18 2022] DELETE /identity/v3/users/ad623d8db0654cda8f9937d33af849aa => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:43:18.615202 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:43:18.617043 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:43:18.630465 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f679a-bc7e-4e34-b1fa-4cc8e30ca21e admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:43:18.688192 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505af82e-8045-432a-82e6-910225ee5a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:43:18.688821 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505af82e-8045-432a-82e6-910225ee5a81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:43:18.689099 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505af82e-8045-432a-82e6-910225ee5a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:43:18.689371 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505af82e-8045-432a-82e6-910225ee5a81 admin admin] PATH_INFO: `/v3/projects/64c5f0c44e604538a9a7397d21a5e814` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:43:18.690470 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505af82e-8045-432a-82e6-910225ee5a81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64c5f0c44e604538a9a7397d21a5e814)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:43:18.692387 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505af82e-8045-432a-82e6-910225ee5a81 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:43:18.739578 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-505af82e-8045-432a-82e6-910225ee5a81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c5f0c44e604538a9a7397d21a5e814', 'request_id': 'req-505af82e-8045-432a-82e6-910225ee5a81'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:18.745164 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-505af82e-8045-432a-82e6-910225ee5a81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c5f0c44e604538a9a7397d21a5e814', 'request_id': 'req-505af82e-8045-432a-82e6-910225ee5a81'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:18.752385 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-505af82e-8045-432a-82e6-910225ee5a81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c5f0c44e604538a9a7397d21a5e814', 'request_id': 'req-505af82e-8045-432a-82e6-910225ee5a81'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:18.763963 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-505af82e-8045-432a-82e6-910225ee5a81 admin admin] The token cache is being invalidate because project 64c5f0c44e604538a9a7397d21a5e814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:43:18.764254 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-505af82e-8045-432a-82e6-910225ee5a81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:18.764886 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-505af82e-8045-432a-82e6-910225ee5a81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:43:18.766360 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1455/2881] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:43:18 2022] DELETE /identity/v3/projects/64c5f0c44e604538a9a7397d21a5e814 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:43:28.438668 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=10,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:43:28.761060 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=77,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:44:16.732844 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab6ed9-4d6b-42b7-b010-a53d995ad240 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:44:16.734426 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-11ab6ed9-4d6b-42b7-b010-a53d995ad240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:44:16.796973 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab6ed9-4d6b-42b7-b010-a53d995ad240 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:44:16.847292 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d15e0f-ecaf-4d62-b85a-59074e15b120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:44:16.847931 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d15e0f-ecaf-4d62-b85a-59074e15b120 service neutron] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:44:16.848214 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d15e0f-ecaf-4d62-b85a-59074e15b120 service neutron] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:44:16.848489 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d15e0f-ecaf-4d62-b85a-59074e15b120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:44:16.850362 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a7d15e0f-ecaf-4d62-b85a-59074e15b120 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:44:16.861381 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d15e0f-ecaf-4d62-b85a-59074e15b120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:44:16.862830 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d15e0f-ecaf-4d62-b85a-59074e15b120 service neutron] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:44:16.874450 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1427/2882] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:44:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:44:17.958178 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505af82e-8045-432a-82e6-910225ee5a81 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:44:17.959764 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-505af82e-8045-432a-82e6-910225ee5a81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:44:17.972359 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505af82e-8045-432a-82e6-910225ee5a81 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:44:17.975994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0cd5e4-10f7-40db-aaf9-b4678c4e46d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '34fcb5885325472bbe338cf05d3788f4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:44:17.976631 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cd5e4-10f7-40db-aaf9-b4678c4e46d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:44:17.976911 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cd5e4-10f7-40db-aaf9-b4678c4e46d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:44:17.977183 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cd5e4-10f7-40db-aaf9-b4678c4e46d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:44:17.979830 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cd0cd5e4-10f7-40db-aaf9-b4678c4e46d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:44:18.055295 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0cd5e4-10f7-40db-aaf9-b4678c4e46d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:44:18.056909 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0cd5e4-10f7-40db-aaf9-b4678c4e46d8 service neutron] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:44:18.067897 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1456/2883] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:44:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:44:18.128831 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d15e0f-ecaf-4d62-b85a-59074e15b120 service neutron] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:44:18.130256 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-a7d15e0f-ecaf-4d62-b85a-59074e15b120 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:44:18.198967 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d15e0f-ecaf-4d62-b85a-59074e15b120 service neutron] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:44:18.204210 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70867a5e-2db4-43c1-b955-1335ec21c08d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ba45e3cff74962965b248f9f017f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:44:18.204820 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70867a5e-2db4-43c1-b955-1335ec21c08d service placement] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:44:18.205103 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70867a5e-2db4-43c1-b955-1335ec21c08d service placement] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:44:18.205379 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70867a5e-2db4-43c1-b955-1335ec21c08d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:44:18.207086 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-70867a5e-2db4-43c1-b955-1335ec21c08d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:44:18.219084 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70867a5e-2db4-43c1-b955-1335ec21c08d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:44:18.220721 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70867a5e-2db4-43c1-b955-1335ec21c08d service placement] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:44:18.230761 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1428/2884] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:44:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:44:18.373891 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0cd5e4-10f7-40db-aaf9-b4678c4e46d8 service neutron] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:44:18.375400 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-cd0cd5e4-10f7-40db-aaf9-b4678c4e46d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:44:18.391132 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0cd5e4-10f7-40db-aaf9-b4678c4e46d8 service neutron] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:44:18.396779 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b66c0-67a5-4e08-96ac-1c5e067967b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:44:18.397394 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b66c0-67a5-4e08-96ac-1c5e067967b8 service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:44:18.397679 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b66c0-67a5-4e08-96ac-1c5e067967b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:44:18.397958 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b66c0-67a5-4e08-96ac-1c5e067967b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:44:18.399783 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-443b66c0-67a5-4e08-96ac-1c5e067967b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:44:18.413016 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b66c0-67a5-4e08-96ac-1c5e067967b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:44:18.414728 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b66c0-67a5-4e08-96ac-1c5e067967b8 service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:44:18.426699 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1457/2885] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:44:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:44:28.226327 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=78 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:44:28.422054 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=40 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:45:42.448004 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70867a5e-2db4-43c1-b955-1335ec21c08d service placement] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:42.449488 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-70867a5e-2db4-43c1-b955-1335ec21c08d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:42.463149 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70867a5e-2db4-43c1-b955-1335ec21c08d service placement] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:42.468639 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6529d8-46c7-49e9-8fb9-1cadb71b7182 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ba45e3cff74962965b248f9f017f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:42.469335 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6529d8-46c7-49e9-8fb9-1cadb71b7182 service placement] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:42.469649 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6529d8-46c7-49e9-8fb9-1cadb71b7182 service placement] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:42.469961 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6529d8-46c7-49e9-8fb9-1cadb71b7182 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:42.472011 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-6a6529d8-46c7-49e9-8fb9-1cadb71b7182 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:42.484277 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6529d8-46c7-49e9-8fb9-1cadb71b7182 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:42.485844 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6529d8-46c7-49e9-8fb9-1cadb71b7182 service placement] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:42.495692 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1429/2886] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:45:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:45:50.411784 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b66c0-67a5-4e08-96ac-1c5e067967b8 service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:50.413986 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-443b66c0-67a5-4e08-96ac-1c5e067967b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:50.428891 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b66c0-67a5-4e08-96ac-1c5e067967b8 service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:50.439337 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79845024-ee0d-4a75-bff7-d472876789bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '311dbed3ab0b44a7830a93227ff9ad33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:50.439825 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79845024-ee0d-4a75-bff7-d472876789bd service nova] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:50.440109 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79845024-ee0d-4a75-bff7-d472876789bd service nova] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:50.440383 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79845024-ee0d-4a75-bff7-d472876789bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:50.442186 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-79845024-ee0d-4a75-bff7-d472876789bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:50.457262 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79845024-ee0d-4a75-bff7-d472876789bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:50.458942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79845024-ee0d-4a75-bff7-d472876789bd service nova] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:50.471894 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1458/2887] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:45:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:45:52.491533 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=6 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:45:55.314887 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6529d8-46c7-49e9-8fb9-1cadb71b7182 service placement] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:55.325518 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6529d8-46c7-49e9-8fb9-1cadb71b7182 service placement] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:55.331825 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58e51b-1dc8-4950-a565-51b587188568 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ba45e3cff74962965b248f9f017f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:55.332687 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58e51b-1dc8-4950-a565-51b587188568 service placement] REQUEST_METHOD: `GET` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:55.333002 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58e51b-1dc8-4950-a565-51b587188568 service placement] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:55.333402 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58e51b-1dc8-4950-a565-51b587188568 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:55.335270 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-9c58e51b-1dc8-4950-a565-51b587188568 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:55.348083 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58e51b-1dc8-4950-a565-51b587188568 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:55.349670 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c58e51b-1dc8-4950-a565-51b587188568 service placement] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:55.361219 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1430/2888] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:45:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 12:45:58.303298 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79845024-ee0d-4a75-bff7-d472876789bd service nova] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:58.304960 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-79845024-ee0d-4a75-bff7-d472876789bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:58.376942 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79845024-ee0d-4a75-bff7-d472876789bd service nova] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:58.383603 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949e955-fafd-4b7b-82a3-356798e76fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:58.384655 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949e955-fafd-4b7b-82a3-356798e76fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:58.385093 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949e955-fafd-4b7b-82a3-356798e76fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:58.385547 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949e955-fafd-4b7b-82a3-356798e76fec admin admin] PATH_INFO: `/v3/users/bf01094f0a444f2396426866d134d222` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:58.387354 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949e955-fafd-4b7b-82a3-356798e76fec admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf01094f0a444f2396426866d134d222)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:58.390837 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949e955-fafd-4b7b-82a3-356798e76fec admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:58.446444 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f949e955-fafd-4b7b-82a3-356798e76fec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf01094f0a444f2396426866d134d222'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.447104 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f949e955-fafd-4b7b-82a3-356798e76fec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf01094f0a444f2396426866d134d222'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.455959 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f949e955-fafd-4b7b-82a3-356798e76fec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf01094f0a444f2396426866d134d222'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.464994 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f949e955-fafd-4b7b-82a3-356798e76fec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf01094f0a444f2396426866d134d222'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.481763 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-f949e955-fafd-4b7b-82a3-356798e76fec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf01094f0a444f2396426866d134d222'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.488876 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1459/2889] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:45:58 2022] DELETE /identity/v3/users/bf01094f0a444f2396426866d134d222 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:45:58.705902 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58e51b-1dc8-4950-a565-51b587188568 service placement] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:58.725415 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58e51b-1dc8-4950-a565-51b587188568 service placement] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:58.807319 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1aaaa-e687-422b-944d-9551d12eed19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:58.808189 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1aaaa-e687-422b-944d-9551d12eed19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:58.808472 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1aaaa-e687-422b-944d-9551d12eed19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:58.808745 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1aaaa-e687-422b-944d-9551d12eed19 admin admin] PATH_INFO: `/v3/projects/dcf72fd7c24f4691b49b142f53d81ea0` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:58.809915 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1aaaa-e687-422b-944d-9551d12eed19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcf72fd7c24f4691b49b142f53d81ea0)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:58.811884 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1aaaa-e687-422b-944d-9551d12eed19 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:58.871348 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-10c1aaaa-e687-422b-944d-9551d12eed19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcf72fd7c24f4691b49b142f53d81ea0', 'request_id': 'req-10c1aaaa-e687-422b-944d-9551d12eed19'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.878051 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-10c1aaaa-e687-422b-944d-9551d12eed19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcf72fd7c24f4691b49b142f53d81ea0', 'request_id': 'req-10c1aaaa-e687-422b-944d-9551d12eed19'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.887991 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-10c1aaaa-e687-422b-944d-9551d12eed19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcf72fd7c24f4691b49b142f53d81ea0', 'request_id': 'req-10c1aaaa-e687-422b-944d-9551d12eed19'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.907503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-10c1aaaa-e687-422b-944d-9551d12eed19 admin admin] The token cache is being invalidate because project dcf72fd7c24f4691b49b142f53d81ea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:45:58.912503 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-10c1aaaa-e687-422b-944d-9551d12eed19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.913236 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-10c1aaaa-e687-422b-944d-9551d12eed19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:58.915574 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1431/2890] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:45:58 2022] DELETE /identity/v3/projects/dcf72fd7c24f4691b49b142f53d81ea0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:45:58.920618 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949e955-fafd-4b7b-82a3-356798e76fec admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:58.922472 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-f949e955-fafd-4b7b-82a3-356798e76fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:58.939280 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949e955-fafd-4b7b-82a3-356798e76fec admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:59.008905 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb64f2-85f4-450f-bef8-67803005e473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:59.009525 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb64f2-85f4-450f-bef8-67803005e473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:59.010031 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb64f2-85f4-450f-bef8-67803005e473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:59.010558 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb64f2-85f4-450f-bef8-67803005e473 admin admin] PATH_INFO: `/v3/users/e908095d089d4729b22a5cfb3f9d9c0b` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:59.012035 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb64f2-85f4-450f-bef8-67803005e473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e908095d089d4729b22a5cfb3f9d9c0b)` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:59.016038 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb64f2-85f4-450f-bef8-67803005e473 admin admin] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:59.024258 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:45:59.080877 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-86fb64f2-85f4-450f-bef8-67803005e473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e908095d089d4729b22a5cfb3f9d9c0b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.081535 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-86fb64f2-85f4-450f-bef8-67803005e473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e908095d089d4729b22a5cfb3f9d9c0b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.088546 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-86fb64f2-85f4-450f-bef8-67803005e473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e908095d089d4729b22a5cfb3f9d9c0b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.096557 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-86fb64f2-85f4-450f-bef8-67803005e473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e908095d089d4729b22a5cfb3f9d9c0b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.108501 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.notifications [None req-86fb64f2-85f4-450f-bef8-67803005e473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e908095d089d4729b22a5cfb3f9d9c0b'} {{(pid=81638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.114843 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1460/2891] 10.222.0.58 () {58 vars in 1224 bytes} [Wed Aug 24 12:45:58 2022] DELETE /identity/v3/users/e908095d089d4729b22a5cfb3f9d9c0b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 12:45:59.288916 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1aaaa-e687-422b-944d-9551d12eed19 admin admin] Authenticating user token {{(pid=81637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:45:59.290505 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.fernet_utils [None req-10c1aaaa-e687-422b-944d-9551d12eed19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:45:59.305260 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1aaaa-e687-422b-944d-9551d12eed19 admin admin] Validating token access rules against request {{(pid=81637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:45:59.363140 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebb831-849d-4f26-a3fe-87a127543773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec5a3f02319e4ee5bdd1b0a6b24084c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae0194ad9d2f4f1ebdd6b4e39c18be2c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:45:59.363829 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebb831-849d-4f26-a3fe-87a127543773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:45:59.364114 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebb831-849d-4f26-a3fe-87a127543773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:45:59.364392 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebb831-849d-4f26-a3fe-87a127543773 admin admin] PATH_INFO: `/v3/projects/b41760d63ef94abf96a0b955325cdf56` {{(pid=81637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:45:59.365652 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebb831-849d-4f26-a3fe-87a127543773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b41760d63ef94abf96a0b955325cdf56)` {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:45:59.367560 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebb831-849d-4f26-a3fe-87a127543773 admin admin] RBAC: Authorization granted {{(pid=81637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:45:59.388933 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:45:59.417616 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-ccebb831-849d-4f26-a3fe-87a127543773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b41760d63ef94abf96a0b955325cdf56', 'request_id': 'req-ccebb831-849d-4f26-a3fe-87a127543773'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.424392 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-ccebb831-849d-4f26-a3fe-87a127543773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b41760d63ef94abf96a0b955325cdf56', 'request_id': 'req-ccebb831-849d-4f26-a3fe-87a127543773'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.432291 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-ccebb831-849d-4f26-a3fe-87a127543773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b41760d63ef94abf96a0b955325cdf56', 'request_id': 'req-ccebb831-849d-4f26-a3fe-87a127543773'} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.445906 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-ccebb831-849d-4f26-a3fe-87a127543773 admin admin] The token cache is being invalidate because project b41760d63ef94abf96a0b955325cdf56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 12:45:59.446412 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-ccebb831-849d-4f26-a3fe-87a127543773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.447471 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG keystone.notifications [None req-ccebb831-849d-4f26-a3fe-87a127543773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 12:45:59.449392 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: [pid: 81637|app: 0|req: 1432/2892] 10.222.0.58 () {58 vars in 1233 bytes} [Wed Aug 24 12:45:59 2022] DELETE /identity/v3/projects/b41760d63ef94abf96a0b955325cdf56 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 12:46:09.111366 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:46:09.442510 dv-cinder-848900-5-iscsi devstack@keystone.service[81637]: DEBUG dbcounter [-] [81637] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 12:50:48.521184 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb64f2-85f4-450f-bef8-67803005e473 admin admin] Authenticating user token {{(pid=81638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 12:50:48.523320 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-86fb64f2-85f4-450f-bef8-67803005e473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:50:48.666588 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb64f2-85f4-450f-bef8-67803005e473 admin admin] Validating token access rules against request {{(pid=81638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 12:50:48.724760 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78756dc5-ba15-45d9-9c3b-77e31b465f60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b7ba45e3cff74962965b248f9f017f11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7fb9c0b20eea4e6dab2cb9fe0b9d677b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 12:50:48.725953 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78756dc5-ba15-45d9-9c3b-77e31b465f60 service placement] REQUEST_METHOD: `GET` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 12:50:48.726238 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78756dc5-ba15-45d9-9c3b-77e31b465f60 service placement] SCRIPT_NAME: `/identity` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 12:50:48.726510 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78756dc5-ba15-45d9-9c3b-77e31b465f60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 12:50:48.728270 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.fernet_utils [None req-78756dc5-ba15-45d9-9c3b-77e31b465f60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 12:50:48.739248 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78756dc5-ba15-45d9-9c3b-77e31b465f60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 12:50:48.741879 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78756dc5-ba15-45d9-9c3b-77e31b465f60 service placement] RBAC: Authorization granted {{(pid=81638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 12:50:48.751491 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: [pid: 81638|app: 0|req: 1461/2893] 10.222.0.58 () {60 vars in 1365 bytes} [Wed Aug 24 12:50:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 12:50:58.747215 dv-cinder-848900-5-iscsi devstack@keystone.service[81638]: DEBUG dbcounter [-] [81638] Writing DB stats keystone:SELECT=89 {{(pid=81638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}